Depth reconstruction in additive fabrication

CROSS-REFERENCES TO RELATED APPLICATIONS

0001This application claims the benefit of U.S. Provisional Application No. 62/789,764 filed Jan. 8, 2019, the contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

0002This invention relates to the precision methods and systems used in additive fabrication.

0003Additive fabrication, also referred to as 3D printing, refers to a relatively wide class of techniques for producing parts according to a computer-controlled process, generally to match a desired 3D specification, for example, a solid model. A class of fabrication techniques jets material for deposition on a partially fabricated object using inkjet printing technologies. The jetted material is typically UV cured shortly after it deposited, forming thin layers of cured material.

SUMMARY OF THE INVENTION

0004Certain additive fabrication systems use Optical Coherence Tomography (OCT) to capture volumetric data related to an object under fabrication. The captured volumetric data can be used by a reconstruction algorithm to produce a surface or depth map for a boundary of the object under fabrication (e.g., topmost surface of the object). One way of computing the surface boundary uses an image processing method such as peak detection.

0005Aspects described herein replace a some or all of the intermediate steps in the process of going from raw OCT data to a surface or depth map (sometimes referred to as 2.5D representation) of the scanned geometry with a machine learning model represented as a neural network. The reconstructed surface or depth map can be used for a variety of computer vision applications such as part inspection. The algorithm and system can also be used for a feedback loop in additive manufacturing systems.

0006In a general aspect, a method for determining estimated depth data for an object includes scanning the object to produce scan data corresponding to a surface region of the object using a first scanning process, configuring an artificial neural network with first configuration data corresponding to the first scanning process, and providing the scan data as an input to the configured artificial neural network to yield the estimated depth data as an output, the estimated depth data representing a location of a part of the object in the surface region.

0007Aspects may include one or more of the following features.

0008The object may include an object under 3D additive fabrication according to a first fabrication process. The first configuration data may correspond to the first scanning process as well as the first fabrication process. The method may include configuring an artificial neural network with first configuration data including selecting said first configuration data from a set of available configuration data each associated with a different scanning and/or fabrication process.

0009The method may include determining expected depth data for the surface region of the object and providing the expected depth data with the scan data to the configured artificial neural network. The expected depth data may include a range of expected depths. Scanning the object may include optically scanning the object at the location on the surface of the object. Scanning the object may include optically scanning the object over a number of locations on the surface region of the object.

0010Scanning the object may include scanning the object using optical coherence tomography. Scanning the object may include processing raw scan data according to one or more of (1) a linearization procedure, (2) a spectral analysis procedure, and (3) a phase correction procedure to produce the scan data. The method may include transforming the scan data from a time domain representation to a frequency domain representation prior to providing the scan data as input to the artificial neural network.

0011The configured artificial neural network may yields a confidence measure associated with the estimated depth data. The method may include providing scan data for a spatial neighborhood associated with the surface region of the object as input to the configured artificial neural network. The spatial neighborhood may include a number of parts of the object in the surface region.

0012In another general aspect, a system for determining estimated depth data for an object includes a sensor for scanning the object to produce scan data corresponding to a surface region of the object using a first scanning process, an artificial neural network configured with first configuration data corresponding to the first scanning process, the artificial neural network configured to determine the estimated depth data representing a location of a part of the object in the surface region. The artificial neural network has one or more inputs for receiving the scan data and an output for providing an estimated depth data.

0013In another general aspect, software stored on a non-transitory computer-readable medium includes instructions for causing a processor to cause a sensor to scan the object to produce scan data corresponding to a surface region of the object using a first scanning process configure an artificial neural network with first configuration data corresponding to the first scanning process, and provide the scan data as an input to the configured artificial neural network to yield the estimated depth data as an output, the estimated depth data representing a location of a part of the object in the surface region.

0014In another general aspect, a method for determining estimated depth data for an object includes scanning the object to produce scan data corresponding to a surface region of the object using a first scanning process, configuring a first artificial neural network with first configuration data corresponding to the first scanning process, configuring a second artificial neural network with second configuration data corresponding to the first scanning process, providing the scan data as an input to the configured first artificial neural network to yield volumetric data representing a location of a part of the object in the surface region, and providing the volumetric data to the configured second neural network to yield the estimated depth data as an output, the estimated depth data representing the location of the part of the object in the surface region.

0015Aspects may include one or more of the following features.

0016The method may include determining expected depth data for the surface region of the object and providing the expected depth data with the scan data to the configured second artificial neural network. The expected depth data may include a range of expected depths.

0017In another general aspect, a method for configuring an artificial neural network for determining estimated depth data for an object includes determining training data including scan data for a number of object and corresponding reference depth data, processing the training data to form configuration data for an artificial neural network, and providing the training data for use in a method for determining estimated depth data.

0018Aspects may have one or more of the following advantages over conventional techniques. The use of an artificial neural network allows for a model can be automatically trained for different scanning processes and materials. As a result, the parameter tuning that may be required in conventional processing pipelines may be avoided. The model may be simpler and require fewer computation steps than conventional processing pipelines (i.e., the model may increase computational efficiency). The model may produce more accurate and higher resolution results.

0019Other features and advantages of the invention are apparent from the following description, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

0020<figref idref="DRAWINGS">FIG. 1</figref> is a 3D printing system.

0021<figref idref="DRAWINGS">FIG. 2</figref> is a workflow for determining volumetric intensity data from raw OCT data.

0022<figref idref="DRAWINGS">FIG. 3</figref> is another representation of the workflow of <figref idref="DRAWINGS">FIG. 2</figref>.

0023<figref idref="DRAWINGS">FIG. 4</figref> is an example of a volumetric intensity profile.

0024<figref idref="DRAWINGS">FIG. 5</figref> is a first workflow for determining surface depth from raw OCT data.

0025<figref idref="DRAWINGS">FIG. 6</figref> is a second workflow for determining surface depth from raw OCT data.

0026<figref idref="DRAWINGS">FIG. 7</figref> is an example of a transformation of a volumetric intensity profile to a surface depth profile.

0027<figref idref="DRAWINGS">FIG. 8</figref> is a first, fully neural network-based workflow for determining surface depth from raw OCT data.

0028<figref idref="DRAWINGS">FIG. 9</figref> is a second, partially neural network-based workflow for determining surface depth from raw OCT data.

0029<figref idref="DRAWINGS">FIG. 10</figref> is a third, partially neural network-based workflow for determining surface depth from raw OCT data.

0030<figref idref="DRAWINGS">FIG. 11</figref> is a fourth, partially neural network-based workflow for determining surface depth from raw OCT data.

0031<figref idref="DRAWINGS">FIG. 12</figref> is a fifth, partially neural network-based workflow for determining surface depth from raw OCT data.

0032<figref idref="DRAWINGS">FIG. 13</figref> is a sixth neural network-based workflow for determining surface depth from raw OCT data using expected depth information.

0033<figref idref="DRAWINGS">FIG. 14</figref> is a seventh neural network-based workflow for determining surface depth from raw OCT data using a spatial neighborhood.

0034<figref idref="DRAWINGS">FIG. 15</figref> is an eighth neural network-based workflow for determining surface depth from raw OCT data using expected depth information and a spatial neighborhood.

0035<figref idref="DRAWINGS">FIG. 16</figref> is a ninth neural network-based workflow for determining surface depth from raw OCT data using multiple processing pipelines to process a spatial neighborhood.

0036<figref idref="DRAWINGS">FIG. 17</figref> is a tenth, multiple neural network-based workflow for determining surface depth from raw OCT data.

0037<figref idref="DRAWINGS">FIG. 18</figref> is an eleventh, multiple neural network-based workflow for determining surface depth from a spatial neighborhood of raw OCT data and expected depth information.

0038<figref idref="DRAWINGS">FIG. 19</figref> is a neural network configuration.

DETAILED DESCRIPTION

00001 Additive Manufacturing System Overview

0039The description below relates additive fabrication, for example using a jetting-based 3D printer <b>100</b> shown in <figref idref="DRAWINGS">FIG. 1</figref>. As is described in greater detail below, the printer <b>100</b> includes a controller <b>110</b> that processes data from a sensor <b>160</b> using a sensor data processor <b>111</b> to determine surface data related to an object under fabrication <b>121</b>. That surface data is used as feedback by a planner <b>112</b> to determine future printing operations. The sensor data processor <b>111</b> may use one of any number of the machine learning-based methodologies described herein to determine the surface data more efficiently and/or more accurately.

0040The printer <b>100</b> uses jets <b>120</b> (inkjets) to emit material for deposition of layers on a partially fabricated object. In the printer illustrated in <figref idref="DRAWINGS">FIG. 1</figref>, the object is fabricated on a build platform <b>130</b>, which is controlled to move related to the jets (i.e., along an x-y plane) in a raster-like pattern to form successive layers, and in this example also to move relative to the jets (i.e., along a z-axis) to maintain a desired separation of the jets and the surface of the partially-fabricated object <b>121</b>. As illustrated, there are multiple jets <b>122</b>, <b>124</b>, with one jet <b>122</b> being used to emit a support material to form a support structure <b>142</b> of the object, and another jet <b>124</b> being used to emit built material to form the object <b>144</b> itself. For materials for which curing is triggered by an excitation signal, such as an ultraviolet illumination, a curing signal generator <b>170</b> (e.g., a UV lamp) triggers curing of the material shortly after it is jetted onto the object. In other embodiments, multiple different materials may be used, for example, with a separate jet being used for each material. Yet other implementations do not necessarily use an excitation signal (e.g., optical, RF, etc.) and rather the curing is triggered chemically, for example, by mixing multiple components before jetting, or jetting separate components that mix and trigger curing on the object. Note that in some examples, after the additive deposition is complete, the object may be subject to further curing (e.g., to complete the curing), for example, by further exposing the object to UV radiation.

0041A sensor <b>160</b> is used to determine physical characteristics of the partially fabricated object, including one or more of the surface geometry (e.g., a depth map characterizing the thickness/depth of the partially fabricated object), subsurface (e.g., in the near surface including, for example, 10s or 100s of deposited layers) characteristics. The characteristics that may be sensed can include one or more of a material density, material identification, and a curing state. While various types of sensing can be used, examples described herein relate to the use of optical coherence tomography (OCT) to determine depth and volumetric information related to the object being fabricated.

0042The controller <b>110</b> uses the model <b>190</b> of the object to be fabricated to control motion of the build platform <b>130</b> using a motion actuator <b>150</b> (e.g., providing three degrees of motion) and control the emission of material from the jets <b>120</b> according to the non-contact feedback of the object characteristics determined via the sensor <b>160</b>. Use of the feedback arrangement can produce a precision object by compensating for inherent unpredictable aspects of jetting (e.g., clogging of jet orifices) and unpredictable material changes after deposition, including for example, flowing, mixing, absorption, and curing of the jetted materials.

0043The sensor <b>160</b> is positioned above the object under fabrication <b>121</b> and measures characteristics of the object <b>121</b> within a given working range (e.g., a 3D volume). The measurements are associated with a three-dimensional (i.e., x, y, z) coordinate system where the x and y axes are treated as spatial axes and the z axis is a depth axis.

0044In some examples, the sensor <b>160</b> measures the volume of the object under fabrication <b>121</b> in its own coordinate system. The sensor's coordinate system might be a projective space, or the lens system might have distortion, even if the system is meant to be orthographic. As such, it may be the case that the measured volume is transformed from the coordinate space of the sensor <b>160</b> to the world coordinate space (e.g., a Euclidean, metric coordinate system). A calibration process may be used to establish mapping between these two spaces.

00002 Sensor Data Processing

0045In some examples, the printer <b>100</b> fabricates the object <b>121</b> in steps, or “layers.” For each layer, the controller <b>110</b> causes the platform <b>130</b> to move to a position on the z-axis. The controller <b>110</b> then causes the platform <b>130</b> to move to a number of positions on the x-y plane. At each (x,y,z) position the controller <b>110</b> causes the jets <b>120</b> to deposit an amount of material that is determined by a planner <b>112</b> based at least in part on the model <b>190</b> and a depth map of the object under fabrication <b>121</b> determined by a sensor data processor <b>111</b> of the controller <b>110</b>.

00002.1 Volumetric Intensity Data Determination

0046Referring also to <figref idref="DRAWINGS">FIGS. 2 and 3</figref>, in general, the sensor data processor determines the volumetric intensity data from data measured by the sensor <b>160</b> at each (x, y, z) position visited by the platform <b>130</b>. For example, as the controller <b>110</b> causes the build platform <b>130</b> to move about the x-y plane, the controller <b>110</b> causes the sensor <b>160</b> to measure an interference signal <b>261</b> (also referred to herein as raw OCT data or “scan data”) for a surface region of the object under fabrication <b>121</b> for the current position (x, y, z) position of the build platform <b>130</b>. In some examples, to do so, the sensor <b>160</b> projects low-coherence light onto the object under fabrication <b>121</b> and onto a scanning reference mirror (not shown), as is conventional in OCT. The projected light has a narrow bandwidth, and in each pulse of a series of pulses, the frequency or wavenumber is varied linearly over time during the duration of the pulse. A combination of reflected light from the object under fabrication <b>121</b> and the light reflected from the scanning reference mirror results in the interference signal <b>261</b>, represented in the time domain. Due to the linear variation of the wavenumber, reflection at a particular distance results in constructive interference that varies periodically over the duration of the interference signal <b>261</b>. The sensor data processor <b>111</b> of the controller <b>110</b> processes that interference signal <b>261</b> using a Fourier transform (e.g., a fast Fourier transform, FFT) <b>262</b> to determine volumetric intensity data <b>263</b> representing an intensity of the reflected light along the z axis for the current (x, y) position of the build platform <b>130</b>. Very generally, the signal amplitude at each frequency in the volumetric intensity data <b>263</b> corresponds to the amount of light reflected at each depth measurement point.

0047In some examples, the measured volumetric intensity data for different (x, y, z) positions of the build platform is combined to form a volumetric profile representing material occupancy in a 3D volume of the object built on the platform. For example, for each point P in a 3D volume the volumetric profile specifies whether that point contains material or not. In some examples, the volumetric profile also stores partial occupancy of material for the point, as well as the type(s) of material occupying the point. In some examples, the volumetric profile is represented as a 3D discrete data structure (e.g., 3D array of data). For example, for each point P in the (x, y, z) coordinate space, a value of 1 is stored in the data structure if the point contains material and a value of 0 is stored in the data structure if the point does not contain material. Values between 0 and 1 are stored in the data structure to represent fractional occupancy.

0048To distinguish between different materials, the stored value for a particular point can denote material label. For example, 0—no material, 1—build material, 2—support material. In this case, to store fractional values of each material type, multiple volumes with fractional values may be stored. Since the data is typically associated with measurements at discrete (x, y, z) locations, continuous values can be interpolated (e.g., using tri-linear interpolation).

0049Furthermore, in some examples, the volumetric data in the volumetric profile is also associated with a confidence of the measurement. This is typically a value between 0 and 1. For example, 0 means no confidence in the measurement (e.g., missing data), 1 means full confidence in the data sample. Fractional values are also possible. The confidence is stored as additional volumetric data.

0050Referring to <figref idref="DRAWINGS">FIG. 4</figref>, one example of a volumetric profile <b>464</b> includes volumetric intensity data for a number of (x, y, z) positions in the reference frame of a coin. In the figure, a slice of volumetric intensity data, taken in the x-direction is referred to as a “scan line.” In this example, for each y-axis value, a number of the volumetric data along the x-axis for a single y-axis value is referred to as a “scan line” <b>465</b>. The scan line shows, areas of higher volumetric intensity with lighter shading and areas of lower volumetric intensity with darker shading. Inspection of the scan line provides an indication of a surface geometry of the coin in the region of the scan line (e.g., by identifying a line of highest volumetric intensity across the scan line).

00002.2 Surface Depth Determination

0051In some examples, a surface of an object is reconstructed from the raw OCT data <b>261</b> collected for the object under fabrication <b>121</b>. For example, after computing a volumetric profile for an object, a topmost surface of the object can be reconstructed by finding a most likely depth (i.e., maximum z value) for each (x, y) position in the volumetric profile using a peak detection algorithm.

0052Referring to <figref idref="DRAWINGS">FIG. 5</figref>, in some examples, the volumetric intensity data <b>263</b> is processed by a first peak detection module <b>566</b> to determine a surface depth for the (x, y) position associated with the volumetric intensity data <b>263</b>. In some examples, the first peak detection module <b>566</b> is configured to identify a first peak in the volumetric intensity data <b>263</b> (see <figref idref="DRAWINGS">FIG. 3</figref>) that exceeds a threshold (e.g., a predetermined threshold) and to determine the surface depth <b>567</b> corresponding to the identified first peak. In some examples, the first peak detection module <b>566</b> also determines a confidence level associated with the determined surface depth (e.g., based on a magnitude of the first major peak.)

0053Referring to <figref idref="DRAWINGS">FIG. 6</figref>, in other examples, before being processed by the FFT <b>262</b>, the interference signal <b>261</b> is first processed by a linearization module <b>668</b> to generate a linearized interference signal <b>669</b>, for example, to compensate for non-ideal sweeping of wavenumber over time. This linearized signa is then processed by a phase correction module <b>671</b> to generate linearized, phase corrected data <b>672</b>, for example, to compensate for different dispersion effects on the signal paths of signals that are combined. The linearized, phase corrected data <b>672</b> is then provided to the FFT <b>262</b> to generate the volumetric intensity data <b>263</b>.

0054In some examples, the determined depth data for different (x, y) positions is combined to form a “2.5D” representation in which the volumetric intensity data at a given (x, y) position is replaced with a depth value. In some examples, the 2.5D representation is stored as a 2D array (or multiple 2D arrays). In some examples, the stored depth value is computed from the volumetric intensity data, as is described above. In other examples, the stored depth value is closest occupied depth value from the volumetric intensity data (e.g., if the mapping is orthographic, for each (x, y) position the z value of the surface or the first voxel that contains material is stored). In some examples, material labels for the surface of an object are stored using an additional 2.5D array. In some examples, the reconstructed depth represented in the 2.5D representation is noisy and is filtered to remove the noise.

0055Referring to <figref idref="DRAWINGS">FIG. 7</figref>, one example of a volumetric profile <b>768</b> for an object is shown on the left-hand side of the page and one example of a depth representation <b>769</b> of the same object is shown on the right-hand side of the page.

00003 Neural Network-based Depth Reconstruction

0056In some examples, some or all of the above-described steps in the process determining a 2.5D representation of a scanned geometry are replaced with a machine learning model represented as a neural network. Very generally, the machine learning model is trained in a training step (described below) to generate configuration data. In a runtime configuration, the neural networks described below are configured according to that configuration data.

00003.1 Full Neural Network Processing

0057Referring to <figref idref="DRAWINGS">FIG. 8</figref>, in one example, the raw OCT data (i.e., the interference signal) <b>261</b> for a given (x, y) position is fed to a neural network <b>871</b>. In general, the raw OCT data <b>261</b> is a sampled signal with n samples (e.g., n=100). The neural network <b>871</b> therefore has n inputs. The neural network <b>871</b> processes the raw OCT data and outputs a surface depth value <b>567</b> (and optionally a confidence measure). Optionally, the neural network <b>871</b> can also output the confidence of the its estimate. In this example, the neural network <b>871</b> internally computes operations equivalent to the Fourier Transform and peak detection (among other) operations.

0058Note that the arrangement in <figref idref="DRAWINGS">FIG. 8</figref> does not include linearization and phase correction stages. To the extent that compensation is desirable to deal with non-ideal of the sweeping by wavenumber, or even intentional deterministic but non-linear sweeping, the neural network <b>871</b> may be trained to accommodate such characteristics. Similarly, the neural network may be trained to accommodate dispersion effects, thereby avoiding the need for a phase correction stage.

00003.2 FFT Followed by Neural Network Processing

0059Referring to <figref idref="DRAWINGS">FIG. 9</figref>, in some examples the raw OCT data (i.e., the interference signal) <b>261</b> for a given (x, y) position is first fed to a Fast Fourier Transform (FFT) module <b>262</b> which computes the volumetric intensity data <b>263</b> for the given (x, y) position. The volumetric intensity data is then provided to a neural network <b>971</b>, which processes the volumetric intensity data <b>263</b> outputs a surface depth value <b>567</b> (and optionally a confidence measure). Optionally, the neural network <b>971</b> can also output the confidence of the its estimate. In this example, the neural network <b>971</b> internally computes operation an operation equivalent to peak detection and other operations but leaves the FFT outside of the neural network <b>971</b>. Because the FFT is a highly optimized procedure, it may be more efficient to use the neural network after FFT has been applied. It is noted that, in some examples additional spatial filtering (e.g., over a neighborhood of (x,y) locations) is applied.

00003.3 Linearization and Phase Correction Prior to FFT and Neural Network Processing

0060Referring to <figref idref="DRAWINGS">FIG. 10</figref>, in some examples, the raw OCT data (i.e., the interference signal) <b>261</b> for a given (x, y) position is pre-preprocessed before it is fed to the FFT module <b>262</b>. For example, the raw OCT data <b>261</b> is first processed by a linearization module <b>668</b> to generate a linearized interference signal <b>669</b>, which is then processed by a phase correction module <b>671</b> to generate linearized, phase corrected data <b>672</b>. The linearized, phase corrected data <b>672</b> is provided to the FFT <b>262</b> to generate the volumetric intensity data <b>263</b>. The volumetric intensity data is then provided to a neural network <b>971</b>, which processes the volumetric intensity data <b>263</b> and outputs a surface depth value <b>567</b> (and optionally a confidence measure).

00003.4 Linearization Prior to Neural Network Processing

0061Referring to <figref idref="DRAWINGS">FIG. 11</figref>, in some examples, the raw OCT data (i.e., the interference signal) <b>261</b> for a given (x, y) position is linearized before it is fed to a neural network <b>1171</b>. For example, the raw OCT data <b>261</b> is first processed by a linearization module <b>668</b> to generate a linearized interference signal <b>669</b>. The linearized interference signal <b>669</b> is provided to a neural network <b>971</b>, which processes the linearized interference signal <b>669</b> and outputs a surface depth value <b>567</b> (and optionally a confidence measure). In this example, the neural network <b>1171</b> performs an equivalent of phase correction, a Fourier transform, and peak finding.

00003.5 Linearization and Phase Correction Prior to Neural Network Processing

0062Referring to <figref idref="DRAWINGS">FIG. 12</figref>, in some examples, the raw OCT data (i.e., the interference signal) <b>261</b> for a given (x, y) position is linearized and phase corrected before it is fed to a neural network <b>1271</b>. For example, the raw OCT data <b>261</b> is first processed by a linearization module <b>668</b> to generate a linearized interference signal <b>669</b>, which is then processed by a phase correction module <b>671</b> to generate linearized, phase corrected data <b>672</b>. The linearized, phase corrected data <b>672</b> is provided to a neural network <b>1271</b>, which processes the linearized, phase corrected interference signal <b>672</b> and outputs a surface depth value <b>567</b> (and optionally a confidence measure). In this example, the neural network <b>1171</b> performs an equivalent of a Fourier transform and peak finding.

00003.6 Neural Network Processing Algorithm with Expected Depth or Depth Range

0063In some examples, when an estimate of the surface depth (or depth range) is available, the estimate is provided as an additional input to guide the neural network. For example, when the system has approximate knowledge of the 3D model of the object under fabrication, that approximate knowledge is used by the neural network.

0064In the context of a 3D printing system with a digital feedback mechanism, the expected depth or depth range can be computed in a straightforward manner. In this additive fabrication method, scanning and printing are interleaved. The algorithm can store depth values computed at the previous iteration. It also has access to the information whether the printing method has printed a layer at a given (x, y) position and expected thickness of the layer.

0065Referring to <figref idref="DRAWINGS">FIG. 13</figref>, both the raw OCT data (i.e., the interference signal) <b>261</b> for a given (x, y) position and the expected surface depth (or depth range) <b>1372</b> for the given (x, y) position are provided to a neural network <b>1371</b>. The neural network <b>1371</b> processes the raw OCT data (i.e., the interference signal) <b>261</b> and the expected surface depth (or depth range) <b>1372</b> and outputs a surface depth value <b>567</b> (and optionally a confidence).

00003.7 Neural Network Processing Algorithm with Spatial Neighborhood

0066In some examples, rather than providing the raw OCT data <b>261</b> for a single (x, y) position to a neural network in order to compute the surface depth value for that (x, y) position, raw OCT data for a spatial neighborhood around that single (x, y) position are provided to the neural network. Doing so reduces noise and improves the quality of the computed surface depth values because, for example, it is able to remove sudden jumps on the surface caused by over-saturated data or missing data has access to more information. For example, a 3×3 or 5×5 neighborhood if interference signals can be provided as input to the neural network.

0067Referring to <figref idref="DRAWINGS">FIG. 14</figref>, the raw OCT data (i.e., the interference signal) <b>261</b> for a given (x, y) position and it's neighboring spatial positions (x−1,y), (x+1,y) (x, y−1), and (x, y+1) are provided to a neural network <b>1471</b>. The neural network <b>1471</b> processes the raw OCT data <b>261</b> for the given (x, y) position and the provided spatial neighbors and outputs a surface depth value <b>567</b> for the given (x, y) position (and optionally a confidence).

00003.8 Neural Network Processing Algorithm with Spatial Neighborhood and Expected Depth

0068In some examples, an expected surface depth or surface depth range is provided to a neural network along with the raw OCT data (i.e., the interference signal) <b>261</b> for a given (x, y) position and its neighboring spatial positions. This is possible, for example, when the 3D model of the object under fabrication is approximately known.

0069Referring to <figref idref="DRAWINGS">FIG. 15</figref>, the raw OCT data (i.e., the interference signal) <b>261</b> for a given (x, y) position and it's neighboring spatial positions (x−1,y), (x+1,y), (x, y−1), and (x, y+1) are provided to a neural network <b>1571</b>. The expected depth (or depth range) <b>1572</b> is also provided to the neural network <b>1571</b>. The neural network <b>1571</b> processes the raw OCT data <b>261</b> for the given (x, y) position and the provided spatial neighbors along with the expected depth (or depth range) <b>1572</b> and outputs a surface depth value <b>567</b> for the given (x, y) position (and optionally a confidence).

0070This is also applicable in additive fabrication systems that include a feedback loop. Those systems have access to a 3D model of the object under fabrication, previous scanned depth, print data sent to the printer, and expected layer thickness.

00003.9 Linearization and Phase Correction Prior to Neural Network Processing with Spatial Neighborhood

0071Referring to <figref idref="DRAWINGS">FIG. 16</figref>, in some examples, the raw OCT data (i.e., the interference signal) <b>261</b> for a given (x, y) position and it's neighboring spatial positions (x−1,y), (x+1,y), (x, y−1), and (x, y+1) are processed in separate, individual processing pipelines to determine their corresponding volumetric data <b>263</b>.

0072In each processing pipeline, the raw OCT data (i.e., the interference signal) <b>261</b> pre-preprocessed before it is fed to the FFT module <b>262</b>. For example, the raw OCT data <b>261</b> is first processed by a linearization module <b>668</b> to generate a linearized interference signal <b>669</b>, which is then processed by a phase correction module <b>671</b> to generate linearized, phase corrected data <b>672</b>. The linearized, phase corrected data <b>672</b> is provided to the FFT <b>262</b> to generate the volumetric intensity data <b>263</b>.

0073The volumetric intensity data <b>263</b> for each instance of the raw OCT data (i.e., the output of each of the processing pipelines) is then provided to a neural network <b>1671</b>, which processes the volumetric intensity data <b>263</b> and outputs a surface depth value <b>567</b> (and optionally a confidence). As was noted above, spatial neighborhoods of different footprints/stencils can be used (e.g., 3×3, 5×5, etc.).

00003.10 Dual Neural Network Processing Algorithm

0074In some examples, a series of two (or more) neural networks are used to compute the surface depth for a given (x, y) position. Referring to <figref idref="DRAWINGS">FIG. 17</figref>, the raw OCT data (i.e., the interference signal) <b>261</b> for a given (x, y) position and it's neighboring spatial positions (x−1,y), (x+1,y), (x, y−1), and (x, y+1) are processed in separate, individual processing pipelines to determine their corresponding volumetric intensity data.

0075In each processing pipeline, the raw OCT data <b>261</b> is provided to a first neural network <b>1771</b><i>a</i>, which processes the raw OCT data <b>261</b> to determine volumetric data <b>263</b> for the given (x, y) position and its neighboring spatial positions. The volumetric data <b>263</b> generated in each of the pipelines is provided to a second neural network <b>1771</b><i>b</i>, which processes the volumetric intensity data <b>263</b> from each pipeline and outputs a surface depth value <b>567</b> (and optionally a confidence).

0076In some examples, this approach is computationally efficient. The overall/combined architecture is deeper but less wide. The computation of the first neural network <b>1771</b><i>a </i>is completed and then stored. Both neural networks can be trained at the same time.

0077Referring to <figref idref="DRAWINGS">FIG. 18</figref>, in some examples, a configuration similar to that shown in <figref idref="DRAWINGS">FIG. 17</figref> is modified such that the the second neural network <b>1871</b><i>b </i>also takes an expected depth value or depth range for the given (x, y) position as input. This is specifically useful for using the algorithm for additive fabrication feedback loop or when the 3D model of the scanned object is approximately known.

00004 Neural Network Training

0078Referring to <figref idref="DRAWINGS">FIG. 19</figref>, in some examples the neural networks described above are implemented using one or more hidden layers <b>1981</b> with Rectified Linear Unit (ReLU) activations. For example, the neural network <b>1971</b> is configured with one hidden layer <b>1981</b> for computing a surface depth value from a of a neural network from raw OCT input.

0079Standard loss functions can be used (e.g., L₀, L₁, L₂, L_inf) can be used when training the neural networks. In some examples, the networks are trained using input, output pairs computed with a standard approach or using a temporal tracking approach. Generally, the neural network is trained using a backpropagation algorithm and a stochastic gradient descent algorithm (e.g., ADAM). In this way, the surface values for each (x, y) position are calculated. The values might be noisy and additional processing (e.g., filtering) of the surface data can be employed (as is described above). The computed confidence values can be used in the filtering process.

0080In some examples, the input training data is obtained using a direct dept computation process (e.g., using the scanning methodologies described above), which may optionally be spatially smoothed to suppress noise. In some examples, the input training data is obtained from scans of an object with a known geometry (e.g., a coin with a known ground truth geometry. In yet other examples, the input training data is obtained as either 2.5D or 3D data provided by another (possibly higher accuracy) scanner.

00005 Implementations

0081The printer shown in <figref idref="DRAWINGS">FIG. 1</figref> is only an example, and other printer arrangements that may be used are described for example, in U.S. Pat. No. 10,252,466, “Systems and methods of machine vision assisted additive fabrication,” U.S. Pat. No. 10,456,984, “Adaptive material deposition for additive manufacturing,” U.S. Pat. Pub. 2018/0056582, “System, Devices, and Methods for Injet-Based Three-Dimensional Printing,” as well as in Sitthi-Amore et al. “MultiFab: a machine vision assisted platform for multi-material 3D printing.” ACM Transactions on Graphics (TOG) 34, no. 4 (2015): 129. The above-described estimation of depth data may be integrated into the feedback control process described in co-pending U.S. Pat. Pub. 2016/0023403 and 2018/0169953. All of the aforementioned documents are incorporated herein by reference

0082An additive manufacturing system typically has the following components: a controller assembly is typically a computer with processor, memory, storage, network, IO, and display. It runs a processing program. The processing program can also read and write data. The controller assembly effectively controls the manufacturing hardware. It also has access to sensors (e.g., 3D scanners, cameras, IMUs, accelerometers, etc.).

0083More generally, the approaches described above can be implemented, for example, using a programmable computing system executing suitable software instructions or it can be implemented in suitable hardware such as a field-programmable gate array (FPGA) or in some hybrid form. For example, in a programmed approach the software may include procedures in one or more computer programs that execute on one or more programmed or programmable computing system (which may be of various architectures such as distributed, client/server, or grid) each including at least one processor, at least one data storage system (including volatile and/or non-volatile memory and/or storage elements), at least one user interface (for receiving input using at least one input device or port, and for providing output using at least one output device or port). The software may include one or more modules of a larger program, for example, that provides services related to the design, configuration, and execution of dataflow graphs. The modules of the program (e.g., elements of a dataflow graph) can be implemented as data structures or other organized data conforming to a data model stored in a data repository.

0084The software may be stored in non-transitory form, such as being embodied in a volatile or non-volatile storage medium, or any other non-transitory medium, using a physical property of the medium (e.g., surface pits and lands, magnetic domains, or electrical charge) for a period of time (e.g., the time between refresh periods of a dynamic memory device such as a dynamic RAM). In preparation for loading the instructions, the software may be provided on a tangible, non-transitory medium, such as a CD-ROM or other computer-readable medium (e.g., readable by a general or special purpose computing system or device), or may be delivered (e.g., encoded in a propagated signal) over a communication medium of a network to a tangible, non-transitory medium of a computing system where it is executed. Some or all of the processing may be performed on a special purpose computer, or using special-purpose hardware, such as coprocessors or field-programmable gate arrays (FPGAs) or dedicated, application-specific integrated circuits (ASICs). The processing may be implemented in a distributed manner in which different parts of the computation specified by the software are performed by different computing elements. Each such computer program is preferably stored on or downloaded to a computer-readable storage medium (e.g., solid state memory or media, or magnetic or optical media) of a storage device accessible by a general or special purpose programmable computer, for configuring and operating the computer when the storage device medium is read by the computer to perform the processing described herein. The inventive system may also be considered to be implemented as a tangible, non-transitory medium, configured with a computer program, where the medium so configured causes a computer to operate in a specific and predefined manner to perform one or more of the processing steps described herein.

0085In general, some or all of the algorithms described above can be implemented on an FPGA, a GPU, or CPU or any combination of the three. The algorithm can be parallelized in a straightforward way.

0086A number of embodiments of the invention have been described. Nevertheless, it is to be understood that the foregoing description is intended to illustrate and not to limit the scope of the invention, which is defined by the scope of the following claims. Accordingly, other embodiments are also within the scope of the following claims. For example, various modifications may be made without departing from the scope of the invention. Additionally, some of the steps described above may be order independent, and thus can be performed in an order different from that described.