System and method for measuring quantity of blood component in fluid canister

0001[Cross-reference of related applications] This application claims the interests of US Provisional Patent Application No. 61 / 703,179 filed on September 19, 2012. This application is incorporated in its entirety by citation. The application also claims the interests of US Provisional Patent Application No. 61 / 646,822, filed May 12, 2012, and US Provisional Patent Application No. 61 / 722,780, filed November 5, 2012. Both applications are incorporated in their entirety by citation.

0002This application relates to US Patent Application No. 13 / 544,646, filed July 9, 2012, which is incorporated in its entirety by citation.

0003The application generally relates to the surgical field and, in particular, to novel and informative systems and methods for measuring extracorporeal blood volume in canisters used in surgical practice.

0004Overestimation and underestimation of patient bleeding contributes significantly to the high surgical fees of hospitals, clinics, and other medical facilities. In particular, overestimation of the amount of blood loss in patients may lead to waste of transfusion-grade blood and high surgical costs for medical institutions, leading to blood shortage. Underestimation of patient blood loss is an important contributor to resuscitation and transfusion delays in bleeding events, amounting to billions of dollars each year in preventable patient infections, readmissions and litigation.

0005Therefore, in the field of surgery, there is a need for new and useful methods for measuring the amount of blood components in fluid canisters. The present invention provides such novel and informative systems and methods.

0006<figref num="1">FIG. 1 is a flowchart showing the method of one embodiment.</figref><figref num="2">FIG. 2 is a flowchart showing a modified example of this method.</figref><figref num="3">FIG. 3 is a flowchart showing a modified example of this method.</figref><figref num="4">4A and 4B are graphs showing an embodiment of this method.</figref><figref num="5">5A, 5B, 5C and 5D are graphs showing variations of this method.</figref><figref num="6">6A and 6B are graphs showing a modified example of this method.</figref><figref num="7">7A and 7B are graphs showing a modified example of this method.</figref><figref num="8">FIG. 8 is a graph showing a modified example of this method.</figref><figref num="9">FIG. 9 is a schematic view of the system of one embodiment.</figref>

0007The following description of preferred embodiments of the invention is not intended to limit the invention to these preferred embodiments, but will allow one of ordinary skill in the art to manufacture and use the invention.

00081. Method As shown in FIGS. 1 and 2, a method for estimating the amount of blood components in a fluid canister S100 is: In the image of the canister, the step of recognizing the reference marker on the canister at block S110; to this reference marker at block S120. The step of selecting the region of the image based on; the step of correlating a part of the region selected in block S130 to the fluid level in the canister; and the step of estimating the fluid volume in the canister based on this fluid level in block S140. And; the step of extracting features from the region selected in block S150; the step of comparing the features extracted in block S160 with the blood component concentration in the canister; and the volume estimated in block S170 and the blood component concentration in the canister. It includes steps to estimate the amount of blood components in the canister based on;

0009As shown in Figure 3, one variant of Method S100 is: Block S112 removing the background from the canister image; Block S120 correlates a segment of this image with a portion of the fluid contained in the canister. And; in block S140 the step of estimating the fluid volume in the canister based on this segment; in block S150 the step of extracting color features from the pixels in the segment; in block S160 this color feature in the canister-blood components It includes a step to compare with the concentration; a step to estimate the blood component content in the canister based on the fluid volume estimated in block S170 and the blood component concentration in the fluid canister;

0010Method S100 functions to implement mechanical vision to estimate blood component content in fluid canisters. In general, method S100 analyzes fluid canister images in block S140 to measure fluid volume in the canister, block S160 to measure blood component concentrations, and block S170 to analyze data that can be combined. Obtain the blood component content in the canister. Accordingly, Method S100 describes variations of US Patent Application No. 13 / 544,646 and / or implementation techniques of this application incorporated herein by reference.

0011The blood component may be whole blood, erythrocytes, hemoglobin, platelets, plasma, or leukocytes. However, method S100 can also implement block S180, which describes estimating the amount of non-blood components in a canister based on the estimated volume and concentration of non-blood components in the canister. The non-blood component may be saline, ascites, bile, washed saliva, gastric juice, mucus, pleural fluid, urine, fecal matter, or other patient's body fluids.

0012The fluid canister may be a suction canister implemented in a surgical setting to collect blood or other bodily fluids, or in other medical, clinical, or hospital settings, where the fluid canister is a canister in method S100. It may be translucent or nearly transparent so that the fluid contained therein can be identified and analyzed. The canister may optionally be an autologous blood collection canister, an intravenous blood transfusion bag, or any other suitable blood or fluid-carrying container for postoperative effluent collection or biofluid collection. For example, the canister may be a surgical fluid canister, which is a translucent container configured to hold the fluid, viewed from the wall and the outside of the container along the wall. A canister with a series of horizontal fluid volume indicators; and an anti-glare strip on the outer surface of the wall. The anti-glare strip can be placed on the canister such that the area selected from the canister image in block S120 includes at least a portion of the anti-glare strip. The anti-glare strip is therefore placed on the canister to reduce glare on the container portion corresponding to the selected area of the image to reduce errors in the estimated content of blood components in the canister caused by glare. Can be done. The anti-glare strip may be an adhesive strip such as 3M's Scotch tape, or it may be a marking printed on the outer surface of a surgical fluid canister. This anti-glare strip has a dull, matte, satin, or other suitable anti-glare surface finish. The anti-glare strip may be a narrow strip extending from near the bottom of the surgical fluid canister to near the top of the surgical fluid canister, but may be of any other shape, geometry, material, or surface finish. , It may be applied to a surgical fluid canister in any way. However,The fluid canister may be another suitable type of canister having other suitable characteristics.

0013The blood and non-blood fluids described above can be recovered in a fluid canister in any amount and concentration during surgery or other medical events, and the fluid content and concentration are substantial with the canister volume memory alone. Since it cannot be estimated in real time, Method S100 can be used to quantify the amount and / or concentration of blood components (eg hemoglobin) and / or other fluids (eg saline). Furthermore, since the volume of extracorporeal blood in the fluid canister can be estimated from the data thus obtained, it is specifically incorporated in this application by reference to surgical sponges, surgical towels and / or others. Substantial comprehensive bleeding monitoring is possible when implemented with any of the methods S100 described in US Patent Application No. 13 / 544,646 for estimating extracorporeal blood volume on the surface of the body.

0014Method S100 can be implemented as a fluid canister analyzer by a computer system to analyze photographic images and estimate the contents of the fluid canister. The computer system may be a cloud-based system (eg, Amazon EC2 or EC3), a mainframe computer system, a grid computer system, or any other suitable computer system. Method S100 is therefore implemented by a handheld (eg, mobile) computer device such as a smartphone shown in FIGS. 1 and 4A or a digital music player, tablet computer, and a native blood component analysis application as shown in FIGS. 1 and 4A. Can be executed. For example, a camera integrated with a computer device can acquire an image of a fluid canister, and blocks S110, S120, S130, and the like can be mounted on a processor integrated with the computer device. Additional or alternative, the computer device may communicate with the remote server, such as via a wirelessly connected internet, in which case the server will perform at least some blocks of method S100 and method S100. At least some of the outputs are sent back to the computer for further analysis and / or release to the user. The computer device may also be equipped with or connected to a digital display so that the method S100 can display information to the user (eg, a nurse or anesthesiologist) through this display.

0015Alternatively, the Method S100 is configured to run a fluid canister, a fluid canister stand, a camera, a camera stand configured to support the camera in the vicinity of the fluid canister, a digital display, or a processor configured to run at least part of the Method S100. And / or can be implemented as a stand-alone blood volume estimation system with a communication module configured to communicate with a remote server running at least part of Method S100. In this implementation, the camera is placed in a substantially fixed position with respect to the fluid canister stand, substantially throughout the time the camera is undergoing surgery or other medical event, and / or until the canister is full. , Can be maintained in a suitable position for obtaining an image of the canister. This allows the camera to periodically acquire and analyze images of the fluid canister, such as every 30 seconds or every 2 minutes. This system, which implements Method S100, also communicates with another or more system that implements one or more methods of US Patent Application No. 13 / 544,646 (eg, in Bluetooth). ), It is possible to estimate the total amount of blood loss in a patient, which enables a substantially comprehensive estimation of the extracorporeal blood volume. However, Method S100 can also be implemented in or by other computer systems, computer devices, or combinations thereof.

0016As shown in FIG. 3, one variant of method S100 comprises block S102, which describes acquiring an image of a canister. Block S102 can interface with a camera or other suitable optical sensor to obtain an image of the field of view of the camera or optical sensor. Here, the canister is in the field of view of the camera or optical sensor. As shown in FIG. 5A, block S102 can acquire a static single frame image containing at least a portion of the fluid canister. Alternatively, block S102 can acquire a multi-frame video image containing multiple still images of the fluid canister. This image may be a color image, a black and white image, a grayscale image, an infrared image, a field of view of an optical sensor, a fingerprint of the field of view of an optical sensor, a cloud point, or any other suitable image.

0017In one implementation example, block S102 acquires images of the canister during surgery according to a time schedule such as every 30 seconds or every 2 minutes. Alternatively, the block S102 implements machine vision and / or machine recognition technology to recognize the canister in the field of view of the optical sensor and trigger image acquisition when the canister (or other blood containment item) is detected. You may try to do it. For example, block S102 can acquire an image of the field of view of the canister each time the user grabs the camera (eg, a computer device incorporating the camera) towards the fluid canister. Similarly, block S102 can work with block S140 to obtain an image of the canister if an increase in the threshold of the canister fluid volume is detected. Therefore, the block S112 can automatically acquire an image of the canister based on the change in the fluid volume in the canister or the imaging availability of the canister, as shown in FIGS. 7A and 7B. Over time, fluid recovery in the canister can be tracked. This can be used to map trends in a patient's fluid loss and / or to predict the characteristics of a patient's fluid loss (eg, bleeding). Alternatively, block S102 may acquire canister images by manual input, such as from a nurse, anesthesiologist, etc.

0018In the implementation example described above, block S102 can further guide the user to image acquisition of the fluid canister. For example, as shown in FIGS. 4A and 4B, the block S102 can display an alignment graphic on the display of the computer device, which also functions as a viewfinder for a camera built into the computer device. In this example, block S102 can prompt the user to align the edges of the canister within the field of view of the camera with the alignment graphic displayed on the display, as shown in FIG. 4A. The alignment graphic may include a side, appearance, decal, and point, line, and / or shape (eg, canister shell) of the fluid canister that aligns with the reference marker. Therefore, the block S102 can guide the user to accurately position the canister with respect to the camera (or other optical sensor) in preparing for imaging. This alignment graphic further or alternatively comprises a curve suggesting a perspective view of the fluid canister. This guides the user to position the canister in the preferred direction (eg, vertical and / or horizontal pitch) with respect to and / or distance from the camera. Block S102 also interfaces with a light source or flash system to control canister lighting during canister image acquisition. Block S102 can additionally or alternatively alert the user with an audible or visible alarm if the canister's illumination is inadequate or too low to accurately estimate the fluid content of the canister's blood content. Block S102 therefore enables image acquisition of fluid canisters with very high accuracy, predictable canister placement reproducibility, lighting replenishment, and the like. These also allow for a very accurate and reproducible estimation of blood component content in block S170.

0019Block S102 can time stamp each image of the canister when the canister is full, replaced, and / or from, thereby tracking changes in fluid levels in the canister in method S100. Patients' trends in blood (and fluid) loss, etc. can be further tracked. However, block S102 can otherwise function to capture an image of the canister.

0020Block S110 of method S100 describes a step of recognizing a canister reference marker in a canister image. In general, block S110 functions to recognize markers associated with the canister in the image. By recognizing this marker, the block S110 can analyze a specific part of the image in a continuous block. Block S110 can recognize the reference marker by implementing suitable machine visual field technology and / or opportunity learning technology. For example, block S120 can be used for object positioning, segmentation (eg edge detection, background removal, grab cut-based algorithms, etc.), physical matching, clustering, pattern recognition, template matching, feature extraction, and histogram extraction (for example. For example, texton map extraction, color histogram, HOG, SIFT, MSER (maximum stable pole region for removing blob features from selected regions), etc.), feature dimension reduction (eg PCA, K-Means, linear) Discriminant analysis, etc.), feature selection, thresholding, positioning, color analysis, parametric regression, non-parametric regression, unsupervised parametric or non-parametric regression, or any other type of opportunity learning or machine vision implemented to implement canister physics Target dimensions can be estimated. Block S110 also includes changing canister lighting conditions, changes in canister fluid components (eg, significantly changing colors, transparency, index of refraction), lens-based or software-based optical distortion in images, or scenarios of use. Other discrepancies or various epidemics can be complemented.

0021In one implementation example, the block S110 can recognize the reference marker which is the boundary between the canister and the background, and the block S110 can remove the image portion corresponding to the background. In another implementation example, as shown in FIG. 5B, block S110 recognizes a reference marker, which is a symbol provided on the canister. For example, this symbol may be a manufacturer's label printed on the canister, a scale of fluid volume printed on the canister, colored dots affixed to the outside of the canister (eg, sticker), various manufacturer's marks. May be a common (eg, standardized) mark printed on a surgical canister, or any other suitable reference marker. In a further implementation example, block S110 recognizes markers based on the surface finish of the canister. For example, block S110 can recognize markers that are part of the canister that includes a matte finish or other virtually glare-free finish.

0022Block S110 can further or alternatively implement mechanical field of view technology to recognize the type of fluid canister. For example, block S110 can implement template matching to detect each reference marker by accessing the reference marker template library. Each reference marker is associated with a canister of a particular type, size, and / or shape from a particular manufacturer. In this implementation example, more subsequent series of blocks in Method S100 can be fitted to a particular type of fluid canister, in which case block S110 is specific for consecutive blocks, depending on that particular canister type. Works to set the implementation path for.

0023Block S112 of method S100 describes the steps to remove background from the image of the canister. Since the background is unlikely to contain useful information related to fluid volume and / or quantity in the fluid canister, block S112 eliminates substantially unnecessary parts of the image, followed by the block of method S100. Now we can focus on the analysis of the parts of the image that are likely to contain information related to the quality and quantity of fluid in the canister, as shown in Figures 5B and 5C. In one implementation, block S112 applies mechanical vision such as edge detection, grab cut, foreground detection, or other suitable technique to crop and detect a portion of the image associated with the physical canister. You can truncate the remaining image of the canister on the outside.

0024In another implementation example, the reference marker recognized by the block S112 is used to fix the periphery of the canister defined in advance to the image. Block S112 can then truncate the image area outside its pre-defined canister perimeter. For example, block S112 may select a particular pre-defined canister-shaped boundary depending on the size and / or geometry of the canister recognized in the image in block S110. Alternatively, block S112 can receive input from a user who is aware of the type of fluid canister and then apply a pre-defined boundary filter depending on the type of canister entered. However, block S112 can otherwise function to remove the background portion from the image of the fluid canister.

0025Block S120 of method S100 describes selecting an area of the image based on a reference marker, as shown in FIG. 5D. Generally, in the block S120, a specific area of the image corresponding to the target specific area on the surface of the canister is selected. Areas of interest are areas with substantially low glare (including, for example, matte coatings or non-glare stickers or tapes), areas closest to the display surface of the camera, approximately center between the sides of the recognized canister. It may be a particular feature of the contents of the canister, such as an area in and / or a canister area with virtually no additional markings, labels, etc. The selected region can further divide the fluid surface in the canister into two, and block S130 can recognize the fluid level in the canister based on the analysis of the selected region. Thus, this selected area may be one or more adjacent and / or non-adjacent pixels in the image, and may include substantive feature information of the contents of the canister. In addition, the selected region may correspond to an opaque canister surface (eg, vertical white stripes) that removes background noise but is sufficient to produce a substantially abrupt color transition near the fluid surface. This makes it possible to estimate the fluid height of the canister at block B130.

0026In one exemplary implementation, as shown in FIGS. 5C and 5D, block S110 mounts the machine field of view to recognize the reference marker placed at the standard position of the canister, and block S120 targets the reference marker and canister. The area of the image is selected according to the standardized distance to the region, and the selected image region corresponds to the region of interest on the canister. For example, block S120 has 20 pixel widths and 100 pixel heights with a geometric center of 50 pixels along the x-axis of the image from the reference marker (eg, from a given center pixel of the reference marker). , Select an image area that is 70 pixels off the y-axis.

0027In another exemplary implementation, block S110 recognizes a reference marker, which is a volume marker on the canister, and block S120 selects an area of the image, which is a set of pixels near a portion of the image corresponding to the volume marker. In this exemplary embodiment, block S130 recognizes the fluid meniscus in this pixel set and compares this fluid meniscus with a volume marker to estimate the fluid level in the canister. For example, block S120 is 20 pixels wide and 100 pixels high, with the upper right corner of the area offset from the left edge of the volume marker by 10 pixels along the x-axis of the image and 20 pixels along the y-axis. , Select the rectangular part of the image.

0028In yet another exemplary implementation, block S110 recognizes the horizontal volume indicator on the fluid canister and defines the first horizontal endpoint of the area selected in block S120 and the common horizontal endpoint of the volume indicator. .. Block S120 further defines the second horizontal end point of the selected area as the central horizontal coordinates of the pixels associated with the horizontal volume indicator, and the first vertical end point of the selected area is the bottom boundary of the fluid-containing portion of the fluid canister. Recognizes as the second vertical end point of the selected region on the recognized surface of the fluid in the fluid canister. From these four endpoints, block S120 selects and fixes a linear region of the image. This selected region can acquire color information of the image along the total vertical height of the fluid in the fluid canister and approximately in the horizontal direction within the separated image of the fluid-containing portion of the fluid canister. it can.

0029In a further exemplary implementation, block S120 can define a selected region that almost completely overlaps the reference marker recognized by block S110. For example, block S110 can recognize a reference marker that is an anti-glare surface (eg, anti-glare tape) on the canister, and block S120 can define a selected area that almost completely overlaps the reference marker.

0030Block S120 also describes the correlation between a portion of the canister containing the fluid as shown in FIG. 5D and a segment of the image. For example, block S120 recognizes the perimeter of the canister in the image and works with block 130 to recognize the surface of the fluid in the canister. Block S130 then selects a segment (image region) that correlates with the portion of the canister containing the fluid in the image that is adjacent to the canister and adjacent to the fluid surface. Block S120 characterizes the colors of the various pixels within the portion of the image associated with the canister and selects the segment containing the pixels characterized as approximately red (eg, including blood). However, block S120 may otherwise function to select an image region based on a reference marker and / or to correlate a segment of the image with a portion of the canister containing the fluid.

0031Block S130 of Method S100 describes that a portion of the selected region is correlated with the fluid level in the canister. In general, block S130 recognizes the surface of the fluid in the canister and the base of the canister (eg, the lowest part of the fluid in the canister) and from this data the fluid in the canister, as shown in Figure 6A. The level can be estimated. As described above, the selected region can bisect the fluid surface and analyze the selected region in block S130 to recognize the fluid surface. In one exemplary implementation, block S120 can estimate fluid height by adding a parameter function (eg, S-shape) to the intensity profile of the selected region corresponding to the anti-glare strip on the canister. In another exemplary implementation, block S130 associates the pixels in the selected area with a fluid (eg, a substantially red pixel) of the fluid in the canister based on the distribution of the y-coordinates of the correlated pixels. The upper and lower boundaries can be calculated. In this example, block S130 starts at the 95th percentile of the y-coordinate, or block S130 starts at the 99th percentile of the y-coordinate of the associated pixel, and the redness of the two adjacent pixels is pre-populated. This percentile can be reduced until it exceeds a set threshold and does not change. However, block S130 may otherwise function to recognize and / or ignore the "false positive" red pixels that do not correspond to the fluid in the canister.

0032In one exemplary implementation, block S130 characterizes the color of each pixel (eg, the red value of each pixel) along the vertical lines of pixels within the selected area. By scanning the pixel line upwards from the bottom of the pixel line (ie, near the base of the canister), block S130 can recognize the first sharp shift in pixel color, which is It can be correlated with the lower boundary of the fluid surface. By further scanning the pixel line from above (ie, near the top of the canister) down, block S130 can recognize a second sharp shift in pixel color, which is a fluid. It can be correlated with the upper boundary of the surface. At block S130, the level of fluid in the canister can be estimated by averaging the upper or lower boundaries of the fluid surface. Alternatively, block S130 scans the pixel shortening line between the upper and lower boundaries up and / or down for a more subtle pixel color along this line. You can focus on further analysis, such as by recognizing changes. For example, in block S130, subtle illumination of higher pixel pixel color can be associated with the fluid meniscus. In another example, block S130 can improve the resolution of the estimated surface of the fluid by re-analyzing the pixels in a series of shortened pixel lines.

0033Similarly, block S130 analyzes two or more adjacent pixel lines in the selected area and compares (eg, averages) the analysis results of each pixel line to estimate the accuracy of the fluid surface position. Can be raised. For example, block S130 compares the position of the border pixel of one of each pixel line set in the selected area and the curve between the fluid-filled portion of the canister and the empty portion of the canister. Borders can be extracted and at block S130 this curved border can be correlated with the fluid meniscus. Alternatively, block S130 may be used to estimate the fluid meniscus. For example, block S130 can implement a look-up table of meniscus size and geometry, from which canister type, fluid characteristics (eg, red values that correlate with blood, moisture in the canister), You can see the angle between the camera and the canister, the distance between the camera and the canister, the fluid level in the conical canister, and / or other suitable variables.

0034Block S130 can additionally or alternatively analyze a cluster of pixels, such as a 4-pixel x 4-pixel cluster in a 4-pixel wideline pixel within a pixel area. Block S130 can analyze individual clusters, or pixels or overlapping clusters or pixels, and average the characteristics of the pixels in each cluster, such as redness and color characteristics. However, block S130 can also function to recognize the fluid surface in the canister in other ways.

0035In block S130, the lower boundary of the fluid in the canister can be measured by implementing a similar method of comparing pixel features. Alternatively, block S130 can presume that the lower boundary of the fluid is at or near the lower boundary of the measured canister. However, block S130 can also function to recognize the lower boundary of the fluid in the canister in other ways.

0036When block S130 recognizes the upper and lower boundaries of the fluid in the canister, block S130 counts the number of pixels between the lower and upper boundaries at approximately the center of the image portion that correlates with the canister, and so on. Calculate the pixel-based height of the fluid. In block S130, pixel-based distance measurements are then given to the canister type and / or the actual or estimated angle between the camera and the canister, the distance between the camera and the canister, and the geometry of the canister (eg, in the canister base and fluid). Convert pixel values to physical distance measurements (eg, inches, millimeters), depending on (diameter on the surface) and / or other correlated measurements between the canister and the camera or between them. Alternatively, in block S140, pixel-based fluid level measurements may be converted directly into an estimated fluid volume in the canister.

0037Block S120 additionally or alternatively receives a manual input that selects or recognizes a reference marker, and block S130 also additionally or alternatively selects or recognizes the surface or height of the fluid in the canister. be able to. For example, method S100 can implement a manual check to teach or correct the automatic selection of reference markers and / or the estimation of the fluid level of the canister. Blocks S120 and S130 thus implement supervised or semi-supervised machine learning to select reference markers and / or produce continuous samples (ie, one or more canister images). It can be used to improve canister fluid level estimation. However, blocks S120 and S130 can also function to select reference markers and / or estimate the fluid level of the canister in another way.

0038Block S140 of Method S100 describes an estimate of the fluid volume in the canister based on the fluid level. In general, block S140 functions in block S130 to convert the estimated star fluid level to a fluid volume estimate based on the canister type and / or geometry, as shown in FIGS. 1 and 7A. For example, the canister may be one of various types of conical trapezium fluid canisters with different geometries used to collect a patient's fluid in an operating room and / or clinical facility. Therefore, when the user enters the canister type and / or geometry, determines it via machine vision technology, and / or is accessed from the canister type and / or geometry database, block S140 provides a fluid level estimate. It can be converted to a fluid volume estimate. In addition, in an implementation example where block S140 converts a pixel-based fluid level measurement to an actual fluid volume measurement, block S140 also further indicates the actual or estimated angle between the camera and the canister, the actual distance between the camera and the canister. Alternatively, an estimated distance (eg, diameter at the base and fluid surface of the canister) and / or other relevant measurements between the canister and the camera or between them are obtained.

0039In one exemplary implementation, block S110 implements object recognition to measure a particular type of canister in an image, and block S130 recognizes the maximum number of pixels between the estimated surface of the fluid and the estimated bottom of the fluid canister. Then block S140 to access the lookup table for a particular type of canister. This look-up table correlates the maximum number of pixels between the bottom of the canister and the surface of the fluid with the canister fluid volume, inputs the maximum number of pixels calculated in block S130 in block S140, and returns it to the fluid volume in the canister.

0040In another exemplary implementation, machine vision technology (eg edge detection) is implemented in block S120 to measure the shape and / or geometry of the fluid canister, and in block S130 between the surface of the fluid and the bottom of the fluid canister. Recognizes the maximum number of pixels in and converts this number of pixels to fluid-level physical dimensions (eg inches, millimeters) in the R canister. The block S140 then converts the fluid level estimated from the block S130 according to the estimated physical cross section of the canister based on the measured shape and / or geometry of the fluid canister into the estimated layer fluid volume in the canister.

0041In yet another exemplary implementation, block S110 implements mechanical vision technology to recognize fluid level marks printed (or embossed, glued, etc.) on a fluid canister, and block S130 recognizes the fluid surface in the canister. Recognize. Block S140 then estimates the fluid volume in the canister based on the fluid volume and one or more fluid level marks near the fluid surface.

0042Alternatively, at block S140, fluid level measurements can be accessed directly from a fluid level sensor connected to the fluid canister (eg, placed in the canister). Block S140 may also receive manual input for reading the fluid level manual in the canister. For example, method S100 can implement a manual check that teaches or corrects automatic fluid volume measurements. Block S140 can thus implement monitoring or semi-monitoring machine learning to improve canister fluid volume estimates over time. However, block S140 can also function to estimate or access measurements of fluid volume in the canister in other ways.

0043Block S150 of method S100 describes extracting features from the selected region. In general, the block S150 functions to recognize properties that indicate the quality of the fluid in the canister in a selected area of the canister image. For block S160, which implements parameter technology to relate the extracted features to the concentration of blood components in the canister, block S150 extracts this feature from one or more pixels in the selected region. This feature features color (red), color intensity (eg red value), lightness, hue, saturation value, brightness, glossiness, eg red, blue, green, cyan in one or more component spaces. , Magenta, yellow, key component, and / or other color-related values such as Lab component space. Block S150 can additionally or alternatively extract one or more features that are histograms of various color values or color-related values in the pixel set in the selected area. As shown in FIG. 6B, for block S160, which implements parameter technology to correlate the extracted features with the blood component concentration in the canister, block S150 is a pixel cluster in the selected region of the canister containing the fluid. Features that correlate with the part can be extracted. This cluster may be a cluster that can be compared to a template image in a template image library with a known blood component concentration. However, block S150 can extract any other features as appropriate from the pixels at or above the location within the selected area.

0044Therefore, as shown in FIGS. 6A and 6B, block S150 extracts features from multiple pixels in the selected area to determine the (total) height of the selected area, which is associated with a portion of the fluid canister containing the fluid. A feature set can be collected that displays the quality of the fluid with respect to width and / or area. For example, block S150 can segment the selected area into pixel clusters of m pixels × n pixels, and the o × p array of pixel clusters almost fills the selected area. Block S150 then analyzes each pixel cluster and extracts one feature for each pixel cluster. Block S150 also averages or combines features from pixel clusters to extract a single feature representing fluid quality from the selected area. In another example, the area selected in block S150 can be segmented into a non-overlapping single pixel thickness (horizontal) column extending in a direction orthogonal to the full width of the selected area. In this example, block S150 can average the pixel features in each column and extract a single feature from each pixel column. Similarly, in block S150, the selected area can be segmented into a set of three pixel thickness columns extending in a direction orthogonal to the full width of the selected area. Here, the outer single columns of each column set (excluding the bottom and top column sets) share an adjacent column set and average the pixels of each column set. To extract a single feature from the pixel set. In block S150, additional or alternative, the selected area can be a non-overlapping triangular pixel cluster, a five-pixel array of overlapping crosses (shown in Figures 6A and 6B), an overlapping circular pixel cluster, or any other suitable shape. Can be segmented into overlapping and / or individual pixel clusters. Also, from these pixel clusters, one or more types of the same or different features from the pixel set can be extracted. Block S150 is an alternative to the selected area

0045Block S150 additionally or alternatively extracts one or more features from the selected area, as described in U.S. Patent Application No. 13 / 544,646, which is incorporated herein by reference in its entirety. be able to. However, block S150 can function to extract features from the region selected in another way.

0046As described in US Patent Application No. 13 / 544,646, Block S150 further comprises the actual or estimated patient's current intravascular hematocrit, the patient's estimated intravascular hematocrit, and the patient's previous intravascular hematocrit. , Fluid canister weight or direct measurement of canister fluid volume, doctor-estimated canister fluid volume, previous fluid canister fluid volume and / or quality, fluid canister previous fluid volume and / or quality, ambient lighting conditions, Type of fluid volume or other identifier, directly measured fluid properties in the fluid canister, patient's biometric signature, patient's medical history, surgical identity, type of surgery currently being performed, or any other non-imaging feature as appropriate. , Can access non-image features. For example, as described in US Patent Application No. 13 / 544,646, the other blocks of block S160 and / or method S100 then implement any of these non-image features and select regions. Select a template image to compare with the pixel clusters in, select a parametric model or feature to convert the extracted features into blood composition estimates, define an alarm trigger for excess fluid or blood loss, and one or more. The extracted features of can be converted into the quantity or quality of other fluids or solids in the fluid canister. However, Method S100 may implement any of these non-image features to transform, enable, or notify other features of Method S100.

0047As shown in FIG. 2, it is stated that the block S160 of the method S100 correlates the extracted features with the blood component concentration in the canister. As shown in FIG. 3, block S160 also states that color features correlate with blood component concentrations in the canister. In general, the block S160 functions to convert one or more features (eg, color features) extracted from the image of the block S150 into an estimated concentration of blood components in the fluid in the canister. As described above, the blood component may be whole blood, red blood cells, hemoglobin, platelets, plasma, white blood cells, or any other blood component. For example, block S160 can implement parameter analysis techniques and / or non-parameter analysis techniques as described in US Patent Application No. 13 / 544.646 to estimate the blood component concentration of the fluid in the canister.

0048In one implementation, block S150 extracts features from pixel clusters in a selected area of the image, and block S160 uses template images in a template image library of known blood component concentrations for each pixel cluster. Tag with blood volume indication based on non-parameter correlation of pixel clusters. For example, as shown in FIG. 6A, block S150 extracts the color intensity in the red component space from the pixel cluster set, and block S160 implements the K-nearest neighbor method, and each extracted feature is used as the red intensity value of the template image. Can be compared with. In this example, each template image contains a pixel cluster tagged with a known amount of fluid, such as hemoglobin volume or mass per unit volume of fluid, or unit pixel (eg, hemoglobin concentration). Each template image may additionally or optionally include pixel clusters tagged with volume, mass, density, etc. per unit volume of fluid in the canister or per unit pixel of other liquid or solid. .. Once the block S160 recognizes a proper match between a particular pixel cluster and a particular template image, the block S160 can reflect known fluid volume information from the particular template image on the particular pixel cluster. Block S160 then sums, averages, and / or combines pixel cluster tags to estimate and output the total blood component concentration of the fluid in the canister. However, block S160 can correlate extracted features with blood component concentrations within the canister via other suitable non-parameter methods or techniques.

0049In another implementation, block S150 extracts features from pixel clusters within a selected region of the image, block S160 implements a parametric model or function, and tags each pixel cluster by blood component concentration. As described in U.S. Patent Application No. 13 / 544,646, one or more features extracted from one pixel cluster are inserted into the parameter function, and the features extracted from the pixel cluster are directly extracted from the pixel cluster into the blood component. It can be replaced with a concentration. Block S160 then repeats this step for each of the other pixel clusters in the selected area. In one embodiment, the extracted features may include one or more of the color intensity within the red component space, the color intensity within the blue component space, and / or the color intensity within the green component space. In this example, the parameter function may be a mathematical operation or algorithm that correlates this color intensity with the hemoglobin mass per unit fluid volume. As described in US Patent Application No. 13 / 544,646, the reflectance of hemoglobin oxide (HbO2) at a certain light wavelength represents the hemoglobin concentration per unit volume of fluid. Therefore, in another embodiment, block S150 can extract the reflectance value at a specific wavelength for each pixel cluster set in the selected region, and block S160 implements a parametric model to obtain the reflectance value for each pixel cluster set. It can be converted to a hemoglobinometry value. In block S160, the hemoglobin concentration values are then combined to estimate the total (ie, average) hemoglobin concentration in the canister. Furthermore, since the hemoglobin content of moist (hydrated) erythrocytes is usually 35%, the erythrocyte concentration can be estimated from the hemoglobin concentration based on the static estimated hemoglobin content (eg, 35%). In addition, block S150 allows access to newly measured hematocrit values or estimates of the patient's current hematocrit values (US Provisional Patent Application No. 61/646, (As described in No. 822), in block S160, measured or estimated hematocrit values can be implemented to replace the estimated red blood cell concentration with the estimated in vitro blood concentration. However, block S160 implements other parametric and / or nonparametric analysis of a single pixel or pixel cluster in the selected region to determine the concentration of one or more blood components in the fluid in the canister. Can be estimated.

0050Block S170 of Method S100 describes a method of estimating the amount of components in a canister based on the estimated volume and concentration of blood components in the canister. Generally, as shown in FIGS. 7A and 7B, in block S170, the quality of blood components (eg, mass) by multiplying the estimated volume of fluid in the canister by the estimated concentration of blood components of the fluid in the canister. , Weight, volume, number of cells, etc.) It works to calculate. For example, block S170 estimates the red blood cell count in the canister or the total extracorporeal blood volume in the fluid canister. Block S170 also interface with the method described in US Patent Application No. 13 / 544,646 to obtain the estimated blood volume in the canister, surgical gauze sponge, surgical towel, surgical drape, and / or. Combined with the estimated blood volume in the surgical bandage, the total blood loss of the patient is estimated as described in Block S190 below. However, block S170 can also function to estimate the amount of blood components in the canister in another way.

0051As shown in FIG. 3, one variant of method S100 comprises block S180, which extracts a second feature from the selected region and extracts this extracted second feature as a non-blood component in the canister. It describes a method of estimating the amount of non-blood components based on the estimated volume and concentration of non-blood components in the canister in correlation with the concentration. In general, the block S180 implements a method similar to the blocks S150, S160, and / or S170 to estimate the non-blood component content (eg, amount) in the canister. As mentioned above, non-blood components can be saline, ascites, bile, washed saliva, gastric fluid, mucus, pleural fluid, urine, fecal matter, or other patient's body fluids, surgical fluids, particulate matter, or canisters. It may be the substance inside.

0052In one implementation, block S180 is a fluid, similar to blocks S150, S160, and S170, which estimate the blood component content in a fluid canister based on the color characteristics of the fluid in the fluid canister (eg, "redness"). The other color features of the canister are analyzed to estimate the content of other substances in the canister. For example, in block S180, the transparency of the fluid in the canister can be analyzed and the estimated fluid transparency can be correlated with the concentration or content of water or saline in the fluid canister. In another example, block S180 extracts the "yellowness" of the fluid (eg, the color intensity in the yellow component space) and uses this yellowness as the concentration or content of plasma and / or urine in the fluid canister. Can be correlated. Similarly, block S150 can extract the "green" of the fluid (eg, color intensity in the green and yellow component spaces) and correlate this green with the concentration or content of bile in the fluid canister. However, block S180 can also estimate the amount and / or concentration of other fluids, particles, or substances in the fluid canister.

0053As shown in FIG. 3, a variant of method S100 comprises block S190, which describes a method of estimating total patient blood loss based on an estimated blood volume in a canister. For example, in block S190, the estimated blood volume can be added to the estimated blood volume of one or more previous canisters, as shown in FIG. Further, as described above, the block S190 provides the interface of this canister of the block S170 to a surgical gauze sponge, a surgical towel, a surgical drape, and / or as described in US Patent Application No. 13 / 544,646. It can be compared with the estimated blood volume in the surgical dressing. In addition, block S190 adds fluid canister blood volume data to the patient's medical record and triggers a warning when the threshold extracorporeal blood volume estimate is reached, or U.S. Patent Application No. 13/544, As described in No. 646, the current patient's hematocrit can be estimated based on the early patient's hematocrit, fluid IVs, blood transfusion, and total estimated blood loss. Block S190 also allows the patient's intracirculatory blood volume, intracirculatory hematocrit, intracirculatory blood viscosity, and / or intracirculatory red blood cell content to be outside the acceptable window. Can be estimated. For example, in block S190, the current patient's total bleeding volume and the patient's intracirculatory hematocrit are within acceptable boundaries, but at a later specific time (eg, about 5 minutes later) as the bleeding rate increases. It can be inferred that there will be excessive blood loss. Block S160 thus allows the patient to determine the need for subsequent autologous blood transfusions, allogeneic blood transfusions, saline infusions, etc., based on trends in the patient's blood parameters. Block S190 therefore estimates the patient's risk based on the estimated bleeding volume, triggers the administration of blood transfusions, and / or what is necessary or risk for the patient after based on the estimated bleeding volume trend. Can be estimated. However, Block S190 may otherwise maintain a comprehensive estimate of the patient's total bleeding (fluid) volume, such as during surgery or other medical events.

0054As shown in FIG. 3, one variant of method S100 includes block S192. It describes displaying canister analysis results such as total fluid volume, estimated hemoglobin content, red blood cell content, extracorporeal blood volume, etc. within the fluid canister. As shown in FIGS. 1, 7A and 7B, the block S192 can control the augmented reality overlay on the still image or the live video feed of the fluid canister, which is also displayed on the display. For example, method In one implementation where the S100 block is implemented by a mobile acquisition device (smartphone, tablet, etc.), the display integrated with the computer device displays the estimated current blood volume and / or hematocrit volume in the canister. can do. This display also displays the estimated blood volume and / or hematocrit volume of the surgical sponge scanned and in the fluid canister after a predetermined period of time, and / or the past and present in the fluid canister and the surgical sponge scanned. The total estimated blood volume for and in the predicted future can be displayed. Block S192 additionally provides users with a single previous fluid quality and / or volume, multiple previous fluid qualities and / or volumes, and / or trends in fluid quality and / or volume that change over time. Can be notified to. However, block S192 may otherwise function to display relevant fluid canister (and sponge) content information.

00552. System As shown in FIG. 9, the system 100 for estimating the amount of blood components in a fluid canister is: optical sensor 110; processor 120 connected to optical sensor 110; executed by processor 120 to acquire an image of the canister on the optical sensor. A software module 122 that directs the processor 120 to select a region of the image that correlates with a portion of the canister containing the fluid, and in the canister based on the selected region. With a software module that estimates the fluid volume, extracts features from the selected region, and estimates the amount of blood components in the canister based on the extracted features; connected to processor 120 and instructed by software module 122. It is equipped with a display 130 that displays the amount of blood components in the canister.

0056System 100 functions to implement the method S100 described above, where an optical sensor (eg, a camera) implements block S102 to obtain an image of the canister and the processor has blocks S110, S120, described above. Implement S130, S140, S150, S160, S170 and others to estimate the quantity and quality of fluid in a surgical suction canister. The system 100, optical sensors, processors, and displays include and / or function as one or more of the components described in US Patent Application No. 13 / 544,646. Surgeons, nurses, anesthesiologists, gynecologists, doctors, soldiers, or other users can use System 100 to collect fluid in a fluid canister during surgery, childbirth, or other medical events. And / or estimate quality. System 100 detects the presence of blood in the canister, calculates the patient's blood loss rate, estimates the patient's risk level (eg, hypovolemia shock), and / or classifies the patient's bleeding. Can be decided. However, the system 100 can perform other suitable functions.

0057System 100 runs an image-based blood estimation application and can be configured as a handheld (eg, mobile) electronic device such as a smartphone or tablet equipped with an optical sensor 110, a processor 120, and a display 130. Alternatively, the components of the system 100 may be substantially inconspicuous (ie, not contained within a single housing). For example, the optical sensor 110 may be a camera that is placed almost permanently in the operating room, where the camera analyzes canister images (eg, according to method S100) on a local network or remote server (including processor 120). ), A computer monitor, television, or display 130, which is a handheld (mobile) electronic device, accesses it to display the output of the processor 120. However, the system 100 may be of other form or may include other elements.

0058System 100 is a consumer who suffers from hospital equipment such as operating rooms, clinical equipment such as delivery rooms, military equipment such as battlefields, or blood loss due to menorrhagia (severe menstrual bleeding) or nasal bleeding (Chinese). It can be used for a variety of equipment, such as home equipment that helps people monitor. However, the system 100 may be used in other equipment.

0059The optical sensor 110 of the system 100 functions to acquire an image of the canister. The optical sensor 110 functions to implement block S102 of method S100 and can be controlled by software module 122. In one exemplary implementation, the optical sensor 110 is a digital camera that captures a color image of the canister or an RGB camera that captures image components individually in the red, green, and blue fields. However, the optical sensor 110 may be multiple and / or other types of cameras, charge-coupled device (CCD) sensors, complementary MOS (CMOS) active pixel sensors, or other types of optical sensors. However, the optical sensor 110 may function to acquire an image of the canister in another way, either in the appropriate format or across the appropriate visible or invisible spectrum.

0060In one implementation example, the optical sensor 110 is a camera provided in a handheld electronic device. In another implementation, the optical sensor 110 is configured to attach to a battlefield nurse's field helmet, configured to be mounted on a pedestal placed in the operating room, mounted on the ceiling above the operating table. Configured to be mounted on a stand-alone blood volume estimation system including a processor 120, a display 130, and a stage tray supporting a canister for imaging, or placed in or attached to other objects or structures. A camera or other sensor configured in.

0061The software module 122 can also control the optical sensor by, for example, setting the optical sensor 110 to autofocus or autoexposure. Additional or alternative, software module 122 can filter low quality images from the canister and pass selective high quality or sufficient quality images through processor 120 for analysis.

0062Following instructions from software module 122, processor 120 of system 100 receives an image of the canister, estimates the fluid volume in the canister, extracts features from the region of the image that correlates with the fluid volume, and extracts the extracted features. The amount of blood component in the canister is estimated based on the estimated volume and the blood component concentration in the canister by correlating it with the blood component concentration in the canister. Processor 120 can therefore implement the block of method S100 described above according to the instructions from software module 122. Processor 120 can also analyze images from different types of images (eg, still images, streaming images, .MPEG, .JPG, .TIFF) and / or images from one or more individual cameras or optical sensors.

0063The processor 120 is connected to an optical sensor via a wired connection (eg, a trace of a shared PCB) or a wireless connection (eg, a Wi-Fi or Bluetooth connection) to obtain an image or field of view of the optical sensor acquired by the optical sensor 110. You can access the images that you can see inside. In one variant, the processor 120 is located within a handheld electronic device that includes an optical sensor 110 and a display 130. In another variant, processor 120 is part of, or is associated with, a remote server that transmits image data from the optical sensor 110 to remote processor 120 (eg, an internet or local network connection). (Via), the processor 120 estimates the extracorporeal blood volume within at least a portion of the canister by analyzing the image of the canister, and this estimation of the blood component volume is transmitted to the display 130.

0064In one implementation example, as described above, the processor 120 can pair a portion of the canister image with a template image via template matching. This template image is one of the template images in the template image library. For example, the system further comprises a data storage module 160 configured to store a template image library of known concentrations of blood components. In this implementation example, as described above, by comparing the extracted features with the template images in the template image library, the features extracted by the processor can be correlated with the blood component concentration. Alternatively, as described above, processor 120 implements a parameter model to estimate the amount of blood components in the canister based on the features extracted from the image.

0065The software module 122 of the system 100 functions to control the optical sensor 110, the processor 120, and the display 130 to acquire an image of the camera, analyze the image, and display the analysis result. Software module 122 can control the processes of system 100 by running applets, native applications, firmware, software, or other suitable code formats on the processor. Generally, the software module controls the application of the block of method S100 described above, while the software module 122 controls and / or implements other suitable processes or methods in or within system 100. be able to.

0066In one exemplary application, software module 122 is a native application installed in system 100, which is a handheld electronic device such as a smartphone or tablet. When selected from a menu within the operating system running this computer device, software module 122 opens, interfaces with the user to initialize a new case, and controls the optical sensor 110 integrated into the computer device. The display 130 is controlled to take an image, implement a mechanical field of view, execute a mathematical algorithm on the processor, estimate the amount of blood component, and display the estimated amount of blood component. However, the software module 122 may be of any other form or type and may be implemented in other ways.

0067Display 130 of System 100 shows an estimated amount of blood components in the canister. The display 130 may be located within a handheld electronic device (eg, a smartphone, tablet, personal data assistant), which also includes an optical sensor 110 and a processor 120. Alternatively, the display may be a suitable display that can be physically connected to a computer monitor, television screen, or other device. The display 130 can be either an LED, an OLED, a plasma, a dot matrix, a segment, an e-ink, or a retinal manipulation display, a series of warning lights corresponding to an estimated blood component, or any other suitable type of display. You may. Finally, the display 130 can communicate with the processor 120 via either a wired or wireless connection.

0068The display 130 can perform at least block S192 of method S100 by displaying estimates of blood components in the canister and / or multiple canisters. This estimation of blood volume can be expressed in a general format, such as "ccs (cubic centimeter)". As mentioned above, this data can be represented in the form of a dynamic augmented reality overlay on top of the canister's live video stream, which is also displayed on display 130. The image from the optical sensor 110 is transmitted to the display 130 via the processor 120 in near real time. The data are alternative, in tables, charts, or graphs showing cumulative elapsed time estimates of blood components over multiple samples analyzed over time, and at least one of the individual blood volume estimates for each canister. Can be shown. The display 130 can display any of previous images of the canister, warnings such as the patient's risk level (heavy hemorrhagic shock), or the patient's bleeding classification, or suggestions for initiating a transfusion. Any of these data, warnings, and / or suggestions can be displayed on multiple screens or made accessible on one or more displays.

0069A variant of the system 100 further comprises an optical sensor 110, a processor 120, and a handheld housing 140 configured to include a display 130. The handheld housing 140, which includes an optical sensor 110, a processor 120, and a display 130, is a handheld (mobile) electronic device capable of estimating blood volume in one or more canisters in any suitable environment such as an operating room or a delivery room. Can be specified. The housing 140 is made of medical grade material and is suitable for use with the system 100, which is a handheld electronic device, in an operating room or other use or medical facility. For example, the housing may be medical grade stainless steel such as 316L stainless steel, medical grade polymer such as high density polyethylene (HDPE), or medical grade silicone rubber. However, the housing may be made of other materials or combinations of materials.

0070In one variant of System 100, System 100 further comprises a wireless communication module 150 that communicates an estimated amount of blood components in the canister to a remote server configured to store the patient's electronic medical records. The system can also update medical records by estimated blood loss over time, patient risk levels, bleeding classification, and / or other blood criteria. The patient's medical record can therefore be updated almost automatically during medical events such as surgery and childbirth.

0071The systems and methods of the preferred embodiment can be implemented and / or implemented at least partially as a machine configured to receive a computer-readable medium that stores computer-readable support. This instruction is performed by a computer-executable element integrated with the hardware / firmware / software elements of the system, optical sensor, processor, display, system or computer equipment, or any suitable combination thereof. .. Other systems and methods of the preferred embodiment can be implemented and / or implemented at least partially as a machine configured to receive a computer-readable medium that stores computer-readable support. This instruction is performed by computer-executable components integrated with the above-mentioned types of devices and networks and computer-integrated components. Computer-readable media can be stored on suitable computer-readable media such as RAMs, ROMs, flash memory, EEPROMs, optical devices (CD or DVD), hard drives, floppy disks, or suitable devices. it can. A computer-executable component is a processor, but any suitable dedicated hardware device (alternatively or additionally) can perform this instruction.

0072Those skilled in the art who estimate the extracorporeal blood volume in the canister can be recognized from the above detailed description and drawings and the scope of claims, and thus deviate from the scope of the present invention defined in the scope of claims of the present invention. Can be modified and modified without.