{'fraction_non_alphanumeric': 0.028088042049934297, 'fraction_numerical': 0.010767995327785078, 'mean_word_length': 4.806188407332839, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
MultiFab

We have developed a multi-material 3D printing platform that is high-resolution, low-cost, and extensible. The key part of our platform is an integrated machine vision system. This system allows for self-calibration of printheads, 3D scanning, and a closed-feedback loop to enable print corrections. The integration of machine vision with 3D printing simplifies the overall platform design and enables new applications such as 3D printing over auxiliary parts. Furthermore, our platform dramatically expands the range of parts that can be 3D printed by simultaneously supporting up to 10 different materials that can interact optically and mechanically. The platform achieves a resolution of at least 40 μm by utilizing piezoelectric inkjet printheads adapted for 3D printing. The hardware is low cost (less than $7,000) since it is built exclusively from off-the-shelf components. The architecture is extensible and modular -- adding, removing, and exchanging printing modules can be done quickly. We provide a detailed analysis of the system's performance. We also demonstrate a variety of fabricated multi-material objects.


Introduction
Multi-material 3D printing holds the great promise of allowing the automated conversion of 3D models with complex structures, appearances, and properties (e.g., mechanical, electrical, chemical, optical) to physical equivalents. It has the potential to accelerate innovation -engineers and hobbyists will have the power to create objects that have been previously impossible or very difficult to fabricate. It will also give them the opportunity to iterate over their designs inexpensively and quickly. Multi-material 3D printing will also impact the general population by allowing mass customization of personalized products. This technology has a myriad of yet unexplored applications that will inspire future research and stimulate a number of industrial markets. Computer graphics researchers both in academia and in industry share this enthusiasm -they have started contributing significantly to the development of applications and tools for 3D printing [Dong et al. 2010;Bickel et al. 2010;Vidimče et al. 2013;Chen et al. 2013;Tompkin et al. 2013;Skouras et al. 2013].
Unfortunately, current multi-material additive manufacturing systems have severe shortcomings. First, access to this technology is very limited; the price of a multi-material 3D printer is prohibitivea Stratasys Objet Connex is priced at $250K, with materials priced at $500 per kilogram [Stratasys ]. This means that very few labs conducting research in computer graphics can afford these systems. The material library is also limited and proprietary, supporting only UV-cured photopolymers (e.g., even full-color printing is not possible). Furthermore, these printers can simultaneously use, at most, only three different materials. Finally, the hardware and software architectures for current multi-material 3D printers are proprietary and inextensible. The exposed input file format, separate STL files for each material, is limiting, e.g., the per layer raster input format is not exposed although it is internally used. Furthermore, any changes to the underlying hardware and software are virtually impossible.
Taking into account these shortcomings, we have developed a 3D printing platform with the following features: • High-resolution: Our system delivers comparable resolution to current high-end systems (at least 40 µm). This is achieved using machine vision to calibrate the system and provide a feedback loop to correct inaccuracies in 3D prints. Our system is the first 3D printing platform to demonstrate a high-precision feedback loop.
• Low-cost: The current platform is built exclusively from low-cost, commercial off-the-shelf components. The use of a computer vision feedback loop simplifies the design and compensates in software for hardware deficiencies. Our system also pioneers inexpensive and safe LED curing (no other 3D printer we know of uses it) and a simple material feeding system. Our system adapts consumer printheads for polymer 3D printing. This is achieved by optimizing the printhead waveform for each material and each droplet size separately. Our current system has a bill-of-materials of less than $7,000 (refer to the supplementary bill-of-materials), and the estimated cost of the printing materials is around $20 per kilogram.
• Multi-material: Our system dramatically expands the range of parts that can be fabricated by simultaneously supporting up to 10 different materials. These materials can interact optically and mechanically. Our current material library has 15 different materials, and it is continuously growing.
• Extensible: The hardware and software architectures are easily extensible and reconfigurable. Our carriage system allows for reconfiguring the printer by adding and removing different modules quickly and easily. The modular software architecture provides an API for the development and use of new modules.
• Multi-material Raster Input: Our platform allows specifying objects using per layer multi-material raster files. This provides more direct access to the printer hardware, thus expanding the range of printable outputs.
• Integration with Auxiliary Objects: The integrated machine vision system allows combining multi-material 3D printed parts with auxiliary objects.
We start by presenting the relevant work on multi-material 3D printing (Section 2). Then, we describe our hardware (Section 3), software (Section 4), and machine vision (Section 5). Finally, we analyze printer performance and present a variety of applications (Section 6).

Previous Work
We review the related work on multi-material 3D printing systems and applications of multi-material 3D printing.
Multi-material 3D printing hardware: There have been significant efforts in the academic community to build multi-material fabrication platforms. Weiss et al. [L.E. Weiss 1997] uses a combination machining and plasma spraying and welding for material deposition. Stereolithography has been adapted to support multiple materials [Maruo et al. 2001;Inamdar et al. 2006;Han et al. 2010;Choi et al. 2011;Zhou et al. 2013]. This is accomplished by using multiple vats with UV-curable polymers. These systems can provide high resolution, but changing materials for each layer makes the printing process very slow. There have also been efforts to use selective laser sintering with multiple powders [Kumar et al. 2004].
Similarly, powder-based 3D printing [Sachs et al. 1990] supports different binders (e.g., to obtain full-color prints). Multi-material inkjet-based systems have also been developed mainly for printing with biopolymers and for tissue engineering applications [Khalil et al. 2005;Burg et al. 2010]. The Fab@Home [Fab@Home ] project has set a goal to develop an inexpensive, extensible, and multi-material fabrication platform. The current hardware (Model 2) [Lipton et al. 2009] supports printing with multiple syringe-based extruders and provides a library of materials. However, the syringe-based extrusion system achieves relatively low resolution.
On the commercial side, some fused deposition modeling (FDM) 3D printers, such as the MakerBot Replicator 2X [MakerBot ], support dual (or triple) extrusion. While these 3D printers are easier to build, the materials cannot be mixed at high spatial resolution. The powder-based 3D printers developed by Z Corp support full-color 3D printing using different color binders. However, this printing technology can only use plaster as a base material. Stratasys Objet Connex printers [Stratasys ] provide 3D printing with up to three different photopolymer materials using inkjet printheads. More recently, 3D Systems has announced a similar multi-material inkjet printer [ 3D Systems ]. As discussed earlier, the systems are highresolution but also expensive and closed. Patents by Stratasys and 3D Systems describe at a high-level designs for inkjet 3D printing systems. None of these commercial 3D printers uses machine vision system for calibration, 3D scanning, closed feedback loop, and alignment with auxiliary objects.

Applications and Tools for Multi-Material 3D Printing: 3D
printing is an emerging research topic in computer graphics. In this area, researchers have already developed a number of applications that specifically take advantage of printing with multiple materials. They have built processes for the design and fabrication of objects with desired deformation properties [Bickel et al. 2010], objects with desired subsurface scattering [Dong et al. 2010;Hašan et al. 2010], lenticular prints [Tompkin et al. 2013], and actuated deformable characters [Skouras et al. 2013]. There has also been a significant interest in applications of multi-material 3D printing in other fields. Some recent applications include designing and printing co-continuous polymers [Wang et al. 2011], bio-inspired structures [Oxman 2011], printed optics [Willis et al. 2012], deformable soft robots [Hiller and Lipson 2012], tough composites [Dimas et al. 2013], and nanomaterial composites [Campbell and Ivanova 2013].
Recently, general tools for multi-material 3D printing have started being developed. Vidimče et al. [2013] propose a programmable pipeline and architecture for direct specification of multi-material objects. This pipeline outputs multi-material rasters for each layer; this format is supported by our 3D printing platform.  describe a functional specification process that translates high level specifications to multi-material 3D prints. Similarly, the output of this process could potentially be used as an input to our system.

Hardware Architecture
The entire hardware system is controlled by a central computer.
The computer processes layers of the 3D model and commands all subsystems in the printer. The communication between the computer and the subsystems is carried out using 100 Mb/s Ethernet and Universal Serial Bus (USB) protocols. Each subsystem has a dedicated microcontroller responsible for the local control. The main subsystems of the printer include: • Positioning Subsystem: Translates commands from the central computer into directives for motor drivers that control position of the printheads with respect to the build platform.
• Printhead Modules: Each module is comprised of a printhead and its drive electronics. The module receives print data from the central computer using the Ethernet protocol and it synchronizes printing with the positioning subsystem.
• UV-curing Module: Uses UV LEDs to cure deposited materi-129:2 • P. Sitthi-Amorn et al. The printhead modules and the UV-curing module are attached directly on the modular rail. For the feeding system, the material containers, valve arrays, and pressure array are mounted on the modular rail. The 3D scanning module is attached directly to the center of the carriage.
als. It communicates with the central computer using the USB protocol.
• 3D Scanning Module: Includes a camera setup with a Michelson interferometer. It is used for system calibration, print inspection, and depth estimation. This module communicates with the central computer using the USB protocol.
• Material Feeding Subsystem: Feeds printing materials to the printheads and maintains adequate pressure and temperature. The subsystem communicates with the computer using the Ethernet protocol.
The system block diagram is shown in Figure 3. In addition, the printer is built around a metal frame and it is enclosed with UVblocking panels. Next, we describe each of these subsystems in detail. Figure 3: System block diagram of the 3D printer. The central computer controls the entire hardware system, which consists of the positioning subsystem, printhead modules, UV-curing module, 3D scanning module, temperature control subsystem, and material feeding subsystem. The printer's modular design supports the attachment of several printhead modules.

Positioning System
Inkjet 3D printing requires positioning a printhead in three dimensions relative to a build platform: two planar dimensions are required to produce a single layer, and an additional dimension is required to reposition the system to fabricate successive layers. Generating three independent degrees of positioning movement requires a minimum of three degrees of actuation.
The X-axis and Y-axis are arranged in a series configuration to move the printer's carriage in a plane. In particular, the Y-axis carries the X-axis, which actuates the carriage, as shown in Figure 4. Both axes use a timing belt drive for actuation and ride on linear ball bearings. High-torque stepper motors actuate the X-and Y-axis belt drives directly providing more than 2000 DPI in resolution.
In contrast to the X-and Y-axes, the Z-axis only needs to make infrequent small movements. A lead screw drive provides exactly the required combination of features for this task. Three independent screw drives are used to control the Z-axis allowing the platform to tile for leveling and calibration. Each linear screw drive is controlled by a stepper motor with resolution of 32000 DPI. Justin 2013 [Lan ] describes the details of the positioning system design.

Figure 4:
The positioning system of the 3D printer. The X, Y, and Z axes are highlighted in red, cyan, and yellow correspondingly. The carriage is highlighted in green. Notice that the Y-axis carries the X-axis. The Z-axis has three actuators that allow it to tilt the build platform about the X-and Y-axes and to linearly move it along the Z-axis.

Modular Rail and Modules
The printer features a modular rail that allows flexible and easy integration of a variety of modules that provide different functions. In our design, each module can be easily removed from or reattached to the rail. Modules can be positioned at any location along the length of the rail, allowing for modules of differing widths to be mounted minimizing rail space. Next, we describe the modules we have developed.

Printhead Module
Our goal is to design a printhead module that is high-resolution and low-cost. To meet this goal, we adapt piezoelectric drop-on-demand printheads used in consumer 2D printers. In particular, we selected MultiFab: A Machine Vision Assisted Platform for Multi-material 3D Printing • 129:3 the printhead in the Epson Workforce 30 printer. The printhead features 600 DPI resolution, nozzles that can eject droplets ranging from 6 pL to 26 pL in volume, and five independent ink channels.
Piezoelectric printheads operate by applying a voltage change to the piezoelectric material in each chamber. The voltage change deflects the piezoelectric material to change the pressure in the chamber and force ink to eject through the nozzle. Joyce 2014 [Kwan ] describes the details of the printhead module.

UV-curing Module
The UV-curing module is responsible for curing the photopolymer materials. Typically, industrial inkjet 3D printers use gas-discharge lamps to cure photopolymer materials. In this printer, UV LEDs are used to cure photopolymer materials with the purpose of lowering costs, facilitating safe operation, and simplifying the light driving hardware.
The curing system developed for this printer is optimized for functionality, its design informed by photopolymer materials formulated in-house and vice versa. It includes ten high-efficiency 365 nm UV LEDs, an off-the-shelf LED driver, and a fan for cooling the UV LEDs. The UV LEDs are selected for their radiant flux density, low power consumption, and relatively small change in radiant flux over a broad temperature range. Please refer to supplemental document for more details.

Imaging Module
The 3D scanning module we have developed is based on a full-field optical coherence tomography setup similar to Dubois et al. [Dubois et al. 2004]. The setup consists of a Michelson interferometer with a high magnification USB camera and a collimated high-power red LED, as shown in Figure 5. A 50/50 linear plastic beamsplitter is used to split the LED beam. The interference fringes are formed only when both light paths are the same. Polarizers are used at the camera and LED locations to control the light density. The camera focusing distance is the same as the distance from the mirror to the printing surface. The camera is capable of inspecting the printing plane with a resolution of 12.7 µm/pixel. Moreover, we add an array of LEDs mounted between the beam splitter and the printing surface. This allows us to use the same module for 2D imaging. Our imaging setup has the following advantages: • Large scanning depth: Typical Michelson interferometers, such as Dubois et al. [Dubois et al. 2004], fix the location between the camera and the imaging surface. They mechanically translate the mirror or change the light source wavelength in order to obtain interference at different depths. Their depth range is limited by the camera depth of field or the range of the light source wavelengths.
Our system does not have this limitation because the location where the interference occurs is always in focus.
• Versatility: By switching on and off the light sources, the setup can operate as either a 3D scanner or a 2D camera. We use 2D imaging to geometrically calibrate printheads with respect to the printing platform and to align auxiliary non-printed components with 3D prints.
• Ease of assembly: The setup requires a small number of components, all of which can be bought off-the-shelf. We provide the complete details of building and assembling this system in the supplemental material.

Material Feeding Subsystem
The piezoelectric inkjet printheads require precise and accurate control of their internal fluid pressure to properly operate. Fluctuations from the required nominal pressure can cause nozzle misfirings or inconsistent droplet size. Pressure fluctuations are caused by Figure 5: Optical Coherence Tomography 3D scanner. The optical components are arranged in a Michelson configuration. The light source is a collimated, red LED. Polarizers are placed in the LED and camera paths to control the light density. The points in the sample that are at the same distance as the reference mirror cause constructive interference patterns. The platform is moved along the Z axis while the interference patterns are detected by the camera. The scanner features a circular scanning area with a diameter of 15 mm.
hydrostatic pressure changes due to material consumption during printing, and hydrodynamic pressure changes due to disturbances from printer movement and fluid flow pressure drops. To ensure that the fluid pressure at the inlet of the printhead is within the required threshold, we have developed a pressure control system. The overall requirements for the system are: scalability for multi-material and multi-printhead support, independent controllability of fluid pressure within ± 30 Pa. Javier 2014 [Ramos ] provides detailed information on this subsystem.

Frame, Enclosure, and Environment Control Subsystem
The structural frame of the printer supports the motion axes of the printer, houses the electronics, and isolates the interior of the printer from the surrounding environment. The overall size of the printer is 1.2x0.6x1 m (width, length, height). The frame structure is fabricated using aluminum T-slotted framing, which provides adequate structural strength and adaptability. Attached to the frame of the printer are UV-blocking acrylic sheets. These sheets isolate the interior of the printer, which can contain volatiles that are emitted during the printing process, from the environment. The panels also protect individuals nearby from the UV light source used to cure photopolymers within the printer. Two electric fans attached to the rear panel of the printer expel any gases or particulates produced during the printing process through a pair of exhaust ducts.

Material Library
Our system dramatically expands the range of parts that can be fabricated by providing an extensive material library. This library includes materials with a wide range of optical, mechanical and appearance properties. Our material development process consists of an initial formulation, density, surface tension and viscosity measurements, UV-curing tests, and jetting optimization. Then, materials are tested by printing 2D patterns and 3D structures. Properties of all materials are properly characterized. All materials presented in this paper are UV-curable photopolymers. Additional materials such as co-polymers, hydrogels, and solvent-based materials can be adapted to be used within our platform. While we have performed initial experiments with many of these materials, this research is beyond the scope of this paper. Currently, our material library includes a 129:4 • P. Sitthi-Amorn et al.
rigid material (RIG), elastic material (ELA), high refractive index material (HR), low refractive index material (LR), and a support material (SPT). We provide more details in the included supplementary document.

Software Architecture
The majority of the processing for the platform is carried out in the central computer. The central computer runs an application called the Fabricator. The Fabricator takes as an input a 3D model in a multimaterial voxel format, in which each voxel contains the material ID. The application processes the multi-material voxel data layer by layer from bottom to top. For each layer, the application performs the following tasks: 1) it sends the commands to the positioning system, 2) it commands the printhead modules to deposit materials, and 3) it turns on/off the UV-curing module. In addition, the application monitors the pressure and temperature, and it executes the cleaning sequence for the printheads. The operation of the Fabricator is described in detail in Section 4.1. The Fabricator communicates with different modules using both Ethernet and USB protocols. The modules have a dedicated microcontroller and they execute the commands provided by the Fabricator. We describe the applications implemented by each module in Section 4.3.

The Fabricator
The Fabricator takes as input 1) a multi-material voxelized model, 2) the model location on the build platform, 3) the geometric location of the printheads and each of their nozzles, 4) the materials currently loaded in each printhead, and 5) the desired resolution (determined by droplet sizes). The Fabricator outputs 1) droplet firing sequences and droplet sizes for each nozzle in each printhead synchronized to an external sync signal, 2) the commands to move the positioning subsystem, and 3) the commands to turn on/off the light of the UVcuring module. In addition, the Fabricator executes the printhead cleaning sequence (e.g., every 10 minutes) that involves moving each printhead to the designated cleaning position and executing a flushing sequence for each nozzle. The algorithm that converts the voxelized model to the firing sequences for each nozzle proceeds in the following steps: • The input model is translated and rotated to the desired location on the build platform. The appropriate support material is added to the model (e.g., for the raft, below the overhangs, and on the vertical boundaries of the object). The model is resampled to the printing resolution. Note that the positioning system allows only discrete X,Y locations for placing material droplets for each printhead. These locations are not exactly aligned with the resampled volumetric grid (e.g., the orientation of each printhead cannot be precisely set).
• We extract and process each multi-material raster layer from the bottom to the top of the model.
• For each layer, we compute the required X-axis passes (movements along the X-axis while keeping the Y position fixed) necessary to fill all non-empty voxels. In order to compute these passes, we use the following procedure. For each possible Y position of the carriage/printheads, we compute all non-empty voxels in the current layer that can be filled (a voxel can be filled if its material matches the material in the printhead and if the discrete nozzle location falls within the voxel). We pick the Y position that fills the most voxels. We subtract these voxels from the set of voxels that need to be filled. We continue computing the X-axis passes until all non-empty voxels are filled. Note that this process is computed for all materials simultaneously.
• For each X-axis pass, we send the desired movement commands to the positioning subsystem. We also convert the voxels that need to be filled in this pass to the corresponding firing patterns in nozzles. We send these firing patterns to the appropriate printheads.
The presented algorithm provides a good trade-off between speed and quality. Since our software architecture is non-proprietary, we can easily experiment with other alternatives. We have experimented with an algorithm that takes into account the precise location of each droplet with respect to the voxel grid and diffuses the error volumetrically [Cho et al. 2003]. In practice, we have not observed a significant improvement in quality. Each of the external modules is abstracted within the Fabricator using the interface shown in Figure 13 in the Appendix section. Therefore, it is easy to add or replace different modules.

Feedback Loop
Our insight is to integrate the feedback loop as a key component of the printing algorithm. The main goal of this feedback loop is to improve and guarantee the 3D print quality. Overall, the fabricator with the feedback loop proceeds in the following way: • The system prints one layer of a voxelized model.
• We compute a binary mask corresponding to where material is placed in the current raster layer.
• A 3D scanning algorithm computes (Section 5.2) the depth but only within the mask. This mask is progressively smaller and smaller as more layers are printed. We observe that depth variations within the mask should be very small -for an ideal print the depth within the mask should be uniform. The depth variations correspond to the imprecisions of the print.
• Based on this information, we compute an additional correction layer. For the points below the expected depth value (minus some epsilon), we mark the pixels in the correction layer raster. For all other points, we do not mark the pixels in the correction layer. For the marked pixels, the corresponding materials are copied from the previous layer. For pixels that are one layer thickness higher than the expected height, we can inform the fabrication engine that these pixels should not be filled in the next regular layer.
• A correction layer is sent to the printer along with the corresponding material for each pixel.
This process is shown in Figure 6.
The above steps can be executed after printing each raster layer. However, this can lead to long execution times since a 3D scanning pass needs to be performed for each layer. We propose an optimization that performs a 3D scanning pass every k layers (e.g., k = 15). The rest of the feedback loop algorithm remains the same with exception that it might be necessary to insert more than one correction layer (if the depth difference is more than one layer thickness).

Module Applications
We have implemented four different external modules that can be used by the Fabricator: the positioning control module, the printhead MultiFab: A Machine Vision Assisted Platform for Multi-material 3D Printing • 129:5 module, the UV-curing module, and the feeding/temperature control module. All modules but the UV-curing module communicate with the Fabricator using Ethernet (TCP/IP and/or UDP); the UV-curing module communicates with the USB protocol. All modules use a dedicated microcontroller to handle low-level operations. They implement the corresponding interface to the API (see Appendix for details).
The microcontroller for the positioning control module implements the following functions: 1) read the current position, 2) move to the desired location, and 3) wait until the move has completed. In addition, the microcontroller generates the sync signal for the printheads for each step in the X-direction. In this way, the firing of the printheads can be synchronized with the carriage movement.
Since the sync signals start at some predefined X location, counting the sync signals exactly specifies the X location.
The microcontroller for the printhead module internally counts the sync signals in order to precisely determine its current X position. The sync counter can be also reset. The microcontroller implements the print function that determines which nozzles to fire at each sync counter value. The microcontroller has a limited buffer size and therefore the data needs to be streamed from the Fabricator (even for one X-axis pass).
The microcontroller for the UV-curing module implements whether the LEDs should be on/off and the desired intensity. The microcontroller for the feeding and temperatures implements the PID controller for both temperature and pressure.

Machine Vision
The machine vision system is a key component of our multi-material 3D printing platform. The software is implemented as an interactive application running on the central computer. The application controls a 3D scanner, its associated camera, and all other printer modules (e.g., positioning control module, printhead modules). Currently, the application performs four main functions: 1) a geometric calibration of the printheads with respect to the build platform and the positioning system, 2) 3D scanning of printed objects, 3) 3D scanning of auxiliary objects, and 4) an alignment of auxiliary parts with a model to be printed.

Geometric Calibration
The printer calibration procedure has the following steps: • The system levels the build platform with respect to the rail that carries modules. This is accomplished by mounting calibration markers at different positions on the build platform. The orientation of the Z-axis is automatically adjusted using an iterative process until all calibration markers are in focus (i.e., they have the same distance to the camera). For simplicity, we fix the build platform level and the rest of the calibration can be computed entirely in 2D.
• The position of the camera is determined with respect to the build platform coordinate system. This is accomplished by imaging a checkerboard pattern of known size fixed to the build platform. The process computes a homography matrix that provides a mapping from the camera coordinate system to the checkerboard pattern. The platform is moved in the X direction and the same checkerboard pattern is captured. This allows us to compute the mapping matrix from the camera to the build platform coordinate system.
• The position of each nozzle is computed with respect to the build platform coordinate system. A 5x5 grid pattern is printed with a single nozzle and the corresponding image is captured. The cross points of the grid are automatically computed or identified by a user in captured images. (The supplemental video also illustrates this process). The locations of the cross points are transformed to the build platform coordinate system using the homography mapping. This gives the offsets from the cross points to the camera. Then, the average of these offsets is computed to accurately obtain the position of the nozzle in the build platform coordinate system. The process of capturing these transformations for each nozzle is time-consuming; therefore, the transformation is computed for the first and the last nozzle in each row and the transformations are interpolated for the intermediate nozzles. Finally, the nozzle position quality is validated by printing copies of the same grid on top of each other using different nozzles.
• The jetting pattern of each nozzle is calibrated. In order to accomplish this, a horizontal line is printed with each nozzle. If the nozzle does not jet properly, it is removed from the printhead nozzle list. If the nozzle jets properly, any vertical shift from the predicted line location is determined.
For many of the current materials, this calibration process is entirely automated. For some difficult materials (e.g., optically clear materials), the automatic detection of the calibration target is challenging. In order to address this, a user interface for manually selecting the calibration points is provided. Alternatively, the user can change the printing material to a black material for which the automated procedure works reliably.

3D Scanning
3D scanning has been a subject of many academic and industrial research projects. In addition, many practical solutions for manufacturing have been developed. However, performing 3D scanning within a 3D printer posses a set of very unique and challenging problems. Here, we discuss these challenges: • High-resolution: The thickness of a layer in our printer is as low as 10 µm and the feature size is 40 µm or less. This means that 3D scanning has to operate at the resolution of 10 µm.
• Translucency: At the scale of 10 µm, all materials (with exception of metals) become translucent or semi-transparent. This means that traditional machine vision methods such as stereo, structured light scanning, shape from shading, etc. cannot be directly used.
• Specular Materials: Many printing materials are very specular. The majority of machine vision algorithms cannot deal with highly specular materials.
• Optically Clear Materials: Some of our materials are optically clear. Standard machine vision algorithms cannot deal with clear materials.
• Lack of Spatial Detail: In many cases, uniform materials are printed. Therefore, surfaces do not have significant textures. This poses severe challenges for stereo, depth from focus/defocus algorithms.
• Real-time Processing: The computation of the 3D scan has to be performed very fast (it should not take more than a few seconds).
Otherwise, it will significantly influence the performance of the entire system.
Based on these requirements, we have decided to use an optical coherence tomography (OCT) scanner. OCT provides very high resolution (better than 1 µm). It can deal robustly with translucent, specular, and optically clear materials. No texture is required for its operation. The image processing component for the scanning is relatively simple and it can be efficiently computed. Figure 7 shows the processing pipeline of our 3D scanner. In order to obtain a depth map of an object using our 3D scanner, a sequence  Figure 7: Image processing pipeline for our 3D scanner. We compute the 3×3 standard deviation. Next we choose the depth in which the standard deviation is highest. We also keep the maximum standard deviation as the confidence map. We perform the depth correction on the depth map from previous stage. Finally, we stitch the depth maps from all the location weighted by the confidence. We assume that the area where the confidence is below certain threshold is a hole, and fill those holes at the end.
of images is captured while changing the distance of the object to the scanner. This is achieved by moving the build platform in the Z direction at a number of discrete steps (this step size is a variable in our system). At each Z position we take a small number of images (e.g., 6). The overall scanning range depends on the scanned object.
In captured images, a high frequency interference pattern is formed when the distance of the path to the object matches the reference optical path within the scanner. The image processing of the image stack proceeds in the following steps: • For each pixel at each depth value, we compute a standard deviation within its 3x3 neighborhood.
• For each pixel, we find the depth value (e.g., image index) that has the maximum value of the standard deviation. We use this depth as our depth estimate. We also store the maximum standard deviation as the depth confidence value.
• In order to deal with imaging noise or missing data, we add a small amount of spatial regularization. This is achieved using a graph cut minimization [Boykov and Kolmogorov 2004], where the labels correspond to depth values, the data term is a negated standard deviation, and the smoothness term corresponds to 8connectivity.
• The depth values in the previous step correspond to the path length at which the reflection from the sample to the beamsplitter equals to that from the mirror to the beamsplitter. Since the mirror and the beamsplitter may not be perfectly aligned, the computed depth map is distorted. We undo this distortion by applying a per pixel correction map to the computed depth map. We describe how to compute this correction map in Section 5.3.
• The scanning area of our system is about 7 × 7mm. This is not sufficient for most samples. Therefore, we cover the sample area with multiple scanning areas. We capture all corresponding depth maps and we stitch them together to form one larger depth map (Section 5.4).

Depth Calibration
The computed depth map from one scan assumes that the mirror and the beamsplitter assembly in Figure 5 are aligned perfectly. This is generally hard to achieve for this hardware setup. To account for this misalignment we perform a depth correction calibration step. This information is used to correct the depth from the scan in the previous section.
The depth calibration is performed by scanning surface of the platform at two different locations such that the scanning areas are overlapped. Let D l1 and D l2 be the depth maps from such scans. If the system has no misalignment then the depths of the points in the overlapping area will be the same. This is typically not the case. For all the pixels (x i 1 , y i 1 ) in D l1 and (x i 2 , y i 2 ) in D l2 such that (x i 1 , y i 1 ) and (x i 2 , y i 2 ) have the same global coordinate location, we seek D c such that Since all the errors in the system are due to rotation and translation of the mirror-beamsplitter assembly, we can assume that D c can be described as a plane Ax + By +C = 0. We can rewrite the Equation 1 as: We solve for A, B and C using a standard linear solver. D c is calculated by enumerating all the pixel x and y. For all subsequent 3D scans, we subtract per pixel D c value from the output depth.

Depth Map Stitching
In case a sample object is larger than the scanning area, we can scan the depth at different locations and stitch the individual depth maps into one larger depth map that covers the desired area. In this case, the user first chooses a bounding box of the object or the area is provided by as the print layer mask (e.g., when using the system for print correction.) Then, we compute the tiling of the scanning area footprint inside the sample area and we scan the corresponding tiles. We allow some amount of overlap in these scans (typically 5-10%) to reduce the discontinuity around the boundary. Finally, we use the confidence values and perform feathering in order to linearly blend the depth maps [Szeliski 2004]. We found that the discrepancy at the boundary of two scans is typically less than 7 µm, which is less than the our printing resolution target.

Integration with Auxiliary Components
The addition of the machine vision system simplifies the integration of additive manufacturing with components that currently cannot be 3D printed (e.g., consumer electronics, custom integrated circuits).
In particular, our system can be used for high precision alignment of models with existing auxiliary parts. We provide two modes for this alignment. In the first mode, we capture an image or a 3D scan of an auxiliary component on the build platform. Then, we can overlay the model to be printed over this component. We can interactively translate and rotate the model over the image or the depth map of the component to facilitate the alignment. The supplementary video illustrates this process.
In the second mode, we click on a few corresponding points between the model and the component. Based on these 2D point correspondences, we compute the best transformation matrix. In addition, MultiFab: A Machine Vision Assisted Platform for Multi-material 3D Printing • 129:7 Figure 8: Left: User interface for our interactive alignment. Users can easily align models to be printed with auxiliary components. Right: Users can also export the scan of the auxiliary component as a mesh with a preserved coordinate system. This allows users to align models to be printed with auxiliary components using external tools. After alignment the aligned models can be imported back to the print client. Figure 9: A rendering of the 3D chart is on the left. A photograph of a printed chart is shown on the right.
we can export the 3D scanned depth map as a mesh and users may employ external tools to align both models. For example, they can use methods such as the iterative closest point (ICP) to align two surfaces.
Finally, we need to consider what happens when we print the model -typically, two models are not going to align perfectly. In the region where the print protrudes inside the auxiliary component, the printed model is cut away. In the region where the printed model is higher than the object, we automatically fill in the region with the first layer of the printed model.

Results
In this section, we analyze the performance of our imaging system, analyze the effective printer resolution, present a number of applications that are relevant to the computer graphics research, and discuss the limitations of the current hardware and software.

3D Imaging
We first evaluate the performance of the imaging system. In general, our 3D scanning is robust and high-quality. The system can scan a variety of different surfaces (e.g., the scanned circuit board in Figure 8 has a variety of different surfaces including shiny metals and low albedo surfaces). The imaging system has two main parameters: the step size in the Z direction and the number of images taken at each Z location. We have observed that there is no visible improvement in the reconstructed depth when the number of images is 6 or more. Therefore, we always set this number to 6 in our system. We have also analyzed the quality of depth maps as a function of the step size. We scan the same area varying the step size from 0.8 µm to 12.8 µm (see Figure 11). We observe progressively larger and larger discretization results. We also compare the depth map quality to a commercial contact-based scanner based on the Gel-Sight sensor [GelSight ]. We observe that the quality of our depth reconstruction at the smallest step size is comparable to the GelSight system. Furthermore, we conclude that our measurement system has more depth resolution than it is necessary for 3D printing since our layers are 10-20 µm thick. In order to minimize the scanning time,  we set the scanning step size to 6.4 µm.

Print Evaluation
We evaluate the printing resolution by printing a custom calibration chart ( Figure 9) for each of the materials. In this step, we visually evaluate the output quality of each material. We verify that printing using multiple nozzles has the same quality as printing with a single nozzle. We observe that the effective resolution varies according to the height of the features. Specifically, we demonstrate this relationship by evaluating the calibration pattern for a given set of materials. For example, for the rigid material features with heights of 0.25, 1.5, and 3 mm, we observe minimum feature sizes of approximately 200, 300, and 500 µm correspondingly. For the high reflective index material features with heights of 0.25, 1.5, and 3 mm, we observe minimum feature sizes of approximately 200, 350, and 500 µm correspondingly. The minimum distance between features varies according to their size. For the HR material, we have found that for 0.5, 1, and 3 mm tall features, the minimum spacing between features is 100,150, and 500 µm correspondingly. We have found the same results for the rigid material. The maximum angle from the vertical for printing without support material is 3 degrees.
We evaluate the quality of our closed feedback loop by printing a number of different tests. For example, we print a 15 layer slab. We show the acquired depth map in Figure 10a). The surface has an average error of 22 µm from the desired value. Our system generates the correction layer ( Figure 10b) and this correction layer is printed over the slab. Finally, we show a depth map of the slab after printing this layer (Figure 10c). We observe that the average error after the correction is only 5 µm. This is much smaller than the layer thickness.
The print times for the shown example prints are presented in Table 1. The print times vary greatly and they are mainly dependent on: the print size, number of materials used (note the long print time for the multi-color tire), printhead raster speed, and number of passes per layer. The current embodiment of the print process was not optimized for print speed. The print times can be significantly optimized.

Applications
The most exciting part of our platform is the applications it enables.
Here, we present samples of what is possible.
Microlens Arrays: Recent work in the field of computational displays shows that novel and better displays can be built by combining traditional display elements with custom optics and by including computation in the loop [Wetzstein et al. 2011]. However, access to custom designed optical elements is severely limited. We show that our printer is capable of printing custom high-quality microlens arrays. In Figure 12a, we show that custom baffles, printed using optically opaque black material, can be inserted in between optically clear lenslets. The microlens array can be directly placed on 129:8 • P. Sitthi-Amorn et al. We print a flat slab of 15 layers. We acquire the corresponding depth map using our 3D scanning (left). We generate and print a correction layer (middle) based on the depth error. We scan the surface after this correction (right) and observe surface quality corresponding to our printing resolution.  Figure 11: Comparing the quality of 3D scanning as a function of step size. We vary step size from 0.8 µm to 12.8 µm and observe increasing discretization. We compare these scans with a contact-based measurement system (GelSight). For comparison, the GelSight image was manually aligned with the images from the 3D scanner. high-resolution screens to obtain a dynamic light field display.
Optical Fiber Bundles: Similarly, Willis et al. [2012] have recently shown that it is possible to print custom fiber optic bundles using Stratasys Objet Connex printers [Stratasys ]. These bundles can transmit light from planar display devices in order to create custom display elements. Furthermore, they can be also combined with sensing elements (e.g., cameras). However, the quality of these fiber optic bundles is relatively low due to a small difference between the refractive index of the base materials. Figure 12b shows the fiber optic bundles printed with our materials can bend light 90 degrees with no significant leakage to the other side.
Multi-material Meta-materials: By spatially arranging different materials and empty spaces, it is possible to obtain heterogeneous structures that have mechanical properties that go beyond those of the base constitutive materials. Some early work on 3D printing multi-material meta-materials has been demonstrated by Bickel et al. [2010] and Skouras et al. [Skouras et al. 2013]. We show that our printer can fabricate even more complex multi-material metamaterials. For example, we design and fabricate a meta-material with a negative Poisson coefficient. This meta-material, designed by combining rigid and elastic materials, and with empty spaces, expands when stretched (Figure 12e). In addition, we design a multimaterial tire with a honeycomb structure that uses four different materials and empty spaces (Figure 1), including both elastic and rigid materials that have different appearances. We believe our platform will allow computer graphics researchers and computational engineers to validate results of simulations against real fabricated results.
Appearance: Printing models with desired appearances has been a subject of recent research in computer graphics [Dong et al. 2010;Hašan et al. 2010]. Z Corp printers (now 3D Systems) are the only printers capable of full-color 3D printing. However, due to the properties of the base powder material, the printed objects need to be post-processed in order to obtain vivid colors. Here, we show that our printer can mix materials with different base pigments and avoid any post-processing. We print objects with desired texture (Figure 12c), materials with desired subsurface scattering (Figure 1), and direct user-specified materials (Figure 12h).

Goal-based Caustics:
Recent work on goal-based caustics [Papas et al. 2011] demonstrates that it is possible to design a refractive surface that redirects light to form a desired image. Typically, the results are manufactured using CNC milling. We demonstrate that these custom high-quality refractive surfaces can be fabricated using our 3D printer. We can control the UV-curing process of the top surface. In particular, we delay curing by a few seconds in order to produce a smooth surface. We show a printed refractive surface and the obtained caustic image in Figure 12d.

Combining 3D Prints and Auxiliary Components:
It is extremely useful to support printing over existing parts and components. Our print alignment procedure allows accurate positioning of a 3D print with respect to the existing part. We show a privacy screen printed over a smart phone display shown in Figure 12g. The screen is accurately aligned with the boundary of the display. We also show 3D printed lens on the top of a flat LED (Figure 12i). The lens position has been registered with the LED using our interactive alignment process. In addition to the 2D alignment, we can use our 3D imaging system to scan and register a 3D print on the top of an existing surface. Figure 1 shows a holder that is printed encasing a razor. In this example, we have printed two sides of the object. First, we use our 3D scanner to acquire the surface geometry, then we print the first half of the holder. Then, the razor is flipped. We MultiFab: A Machine Vision Assisted Platform for Multi-material 3D Printing • 129:9 scan the surface and align it to the other side of the holder. Finally, we print the second half of the holder.
Printable Fabrics: Measurement, rendering, and simulation of fabrics have been extensively studied by computer graphics researchers [Zhao et al. 2013]. We show that our printer can manufacture thin shells by 3D printing fibers. In our case, the fibers can be made of both elastic and rigid materials and can have different appearances (Figure 12f). Moreover, the support material can be used to ensure that neighboring or crossing fibers are not fused together. This manufacturing method is more general than the traditional warp/weft patterns or various knitting patterns.

Limitations and Discussion
Finally, we discuss the limitations of the current system and the areas for future improvement. Speed: The current system has been optimized for precision but not printing speed. Each layer is approximately 13 µm thick (e.g., 77 layers are required for 1 mm of object) and printing each X pass requires more than 4 seconds. Printing speed could be increased by using larger droplets at the cost of resolution. Loading the same material in all cartridges could also increase printing speed by a factor of 10x; however, we would sacrifice the multi-material capability. Finally, we could increase printing speed by a factor of two by installing an additional UV-curing module. In this way printing could be performed both on the (+) and (-) movement along the X-axis.
Resolution: Our printing resolution matches or exceeds the resolution of high-end commercial 3D printers. We believe this resolution is sufficient for the majority of 3D printing applications. The current resolution limiting factor is the resolution of the printheads and positioning system. In principle, we could add higher resolution printheads ( [Fujifilm Dimatix ]). However, this would further increase printing time. The resolution of our positioning system could also be further improved using an encoder for each of the axes.

Materials:
We provide a significant library of printing materials, but there is virtually unbounded space for the development of materials that can be reliably 3D printed. Examples include exploring the influence of different nano-particles on base materials and developing bio-compatible printing materials.
OCT Scanning: The presented embodiment of the OCT scanner has some practical scanning limitations that limit the scan quality for some types of objects. Specifically, highly specular angled surfaces exhibit a relatively low signal-to-noise ratio. Highly specular surfaces such as mirror-finishes reflect most of the light in their normal direction. For some angled surfaces the normal direction is outside the scanner direction. In practice, a maximum scanning angle of about 5-10 degrees has been observed for highly specular surfaces. For highly diffuse surfaces a maximum angle approaching 90 degrees (vertical surface) has been observed. Additionally, the OCT scanner can only scan depth map profiles, no overhangs or undercuts.

Feedback Loop Print Correction:
The presented correction method is based on an additive process. Material can only be added to areas in the part that are missing material when compared to the input model or reference surface. Removing material would require an additional mechanism (mechanical or chemical) for subtractive correction. This could be an interesting direction for future research.

Conclusions
We have presented a platform for multi-material additive manufacturing. Our platform is high-resolution, low-cost, extensible, and modular. We have shown a large variety of multi-material 3D prints that can be fabricated using this system.
We believe that our work provides multi-material design and fabrication capabilities to the entire computer graphics community as these have been so far severely limited by the expensive and proprietary nature of current systems. By fabricating real materials and objects, we will be able to improve and validate simulation algorithms.
We think that this research will have impact that goes beyond computer graphics. Engineers and hobbyists will be able to design and fabricate a variety of functional devices and objects. Students and teachers will be able to create complex mathematical figures, physics sets, lens systems, and anatomical models.