text:
|Publication number
|Publication type
|Application number
|Publication date
|23 May 2006
|Filing date
|8 Dec 2000
|Priority date
|8 Dec 1999
|Fee status
|Also published as
|Publication number
|James L. Overbeck, Richard J. Van Andel
|Original Assignee
|X-Rite, Incorporated
|Export Citation
|BiBTeX, EndNote, RefMan
|Patent Citations (99), Non-Patent Citations (4), Referenced by (2), Classifications (27), Legal Events (13)
|External Links: USPTO, USPTO Assignment, Espacenet
This application claims the benefit of U.S. Provisional Application No. 60/169,638, filed Dec. 8, 1999 and entitled DENTAL VISION SYSTEM.
The present invention relates to instruments for measuring optical characteristics—for example color, translucency, and/or gloss—of objects and, more particularly, to such instruments for use in dental applications.
The determination of shade or color of an object is a process that is frequently performed in the field of dentistry. To perform dental restorations of a damaged tooth, a dentist visually compares the color of the tooth to be replaced with an assortment of shade tabs. These shade tabs are physical tabs representing the color of commercially available restorative material, such as ceramics. The tabs include the exact specification of materials needed to produce a restorative tooth with the shade of the original tooth as determined by the dentist's visual comparison. Once the dentist finds a shade tab that matches the color of the tooth, or remaining adjacent teeth in some cases, he is in a position to manufacture the required restoration. This process, however, is very time consuming and quite subjective, and frequently results in poorly shaded restorations.
In the field of dentistry, intraoral cameras are frequently used to acquire images of teeth and determine treatment plans for cavities and other mechanical reconstruction. These cameras are designed to be versatile and able to collect measurements in tight places often found in the mouth; however, they do not preserve the color fidelity—that is, they do not collect the true color—of the object measured.
Some dentists attempt to use intraoral cameras to assist in the shade determination process. Unfortunately, conventional intraoral cameras suffer two problems: distance sensitivity due to illumination geometry and color discrimination error due to sensor limitations.
With regard to the first problem, intraoral cameras typically use fiberoptic illumination to reduce the size of the handpiece. Such a device is disclosed in U.S. Re. Pat. No. 36,434 to Hamlin et al, reissued Dec. 7, 1999. The goal of Hamlin, and most intraoral cameras, is to provide a small measuring tip on a handpiece that can be used to probe hard to reach areas in a mouth. Although fiber-optic illumination is useful for providing high levels of illumination and is compatible with small measuring probe tips, a drawback of any small illumination source that illuminates a larger area is that the projected beam must be divergent light. The intensity of a divergent beam is governed by the inverse square law given below:
where I is intensity, D is the distance from the illumination source, and ΔD is an increase in distance D from the light source. The concept of Equation 1 is illustrated in
It is known that when the distance change to the illumination source is significant with respect to the distance to the source, the illumination output varies significantly, creating what is called non-uniform illumination. Particularly with objects positioned close to the fiber optics, certain regions of the object are non-uniformly illuminated because the light from the illumination source rapidly diffuses as it travels away from the source. Moreover, when multiple sources of light are used to illuminate an object, the object may be non-uniformly illuminated in different regions.
An example of non-uniform illumination of the surface of an object is understood with further reference to
An example of non-uniform illumination of regions of an object with a multiple fiber optic light sources is illustrated in
With non-uniform illumination, conventional intraoral cameras critically rely on illumination source positioning which can not be maintained in practical use. This results in significant errors affecting tooth shade determination.
Other devices, specifically designed for tooth shade determination, have been proposed that use bi-directional fiber optic illumination. Such a method is described in U.S. Pat. No. 6,038,024 to Berner, issued Mar. 14, 2000. A limitation of this method of illumination is that the illuminant intensity is maximized at the intersection of the two projected beams. Often, significant portions of the measured area are not illuminated by both beams and hence have a lower and unpredictable illumination value.
Berner's non-uniform illumination is depicted in
Given this non-uniform illumination, a color sensor, sensing the light reflected from the tooth, will invariably collect inconsistent color value information from region to region. For example, what is sensed as "lighter shade" in region 167 may be sensed as "darker shade," in region 169 due to the non-uniform illumination. Moreover, with multiple light source paths, gloss artifact potential is increased. Where glare artifacts exist, the color of the target is washed out by the image of the light source itself rather than the desired tooth subject.
In addition to non-uniform illumination, today's intraoral cameras utilize color filter array (CFA) image sensors that frequently contribute to inaccurate color measurement because the filter array is applied to the image. Many intraoral cameras include color filter arrays such as red, green and blue (RGB) arrays, and cyan, magenta, yellow and green (CMYG) arrays, to name a few. Generally, these color filter arrays are made up of a multitude of adjacent elements called "pixels" (i.e., picture elements). Each pixel measures only the bandwidth of light it is designed to collect. Therefore, in a region of a image corresponding to a pixel, only the bandwidth of light specific to that pixel is displayed, even though the object measured may include other colors in that region.
The operation of and problems associated with color filter arrays are more readily understood with reference to a particular array. A few pixels of a CFA RGB sensor are illustrated in
CFAs do not accurately measure color primarily because of two factors: pixel spacing separation and poor color fidelity. First, the pixel spacing separation factor may be understood with reference to the RGB sensor in
Accordingly, RGB sensors collect only one bandwidth for each point on the tooth even though that point may reflect many bandwidths. As a result, any measurement data for that point will include only data selectively collected by the R,G, or B pixel associated with that point. Moreover, prosthesis manufactured from this measurement data collected with an RGB sensor will not accurately reflect the true color of each point on the tooth. This phenomenon applies to all CFA sensors.
The second factor affecting color measurement is poor color fidelity of CFAs. The mass market for color sensors, in particular CFAs, is consumer electronics and video applications. The goal of such devices is to provide good image resolution, high image acquisition speed and reasonable color fidelity as needed for broadcast and personal imaging applications. CFAs are designed to be inexpensive to manufacture, to provide direct acquisition of RGB data and to provide reasonable low light performance. These design goals come at the cost of color fidelity. More specifically, today's RGB CFA collects selected wavelengths of light impinging on them, but they also incidentally collect unwanted wavelengths in the process. For example, a blue pixel of an RGB array is coated with a polymer that is designed to (a) allow only light of blue bandwidths to be transmitted through the polymer—acting like a filter—and sensed by that pixel, and (b) attenuate all other wavelengths, that is, prevent them from being sensed by that pixel. Typical CFA filters attenuate unwanted wavelengths by only 1/10th of the value of the maximum transmittance of the filter. This lack of rejection of light outside of the wavelengths of interest degrades color fidelity to an unacceptable level for accurate color measurement.
Due to signal detection problems caused by pixel spacing and poor color fidelity, CFA-type sensors are not accurate enough for satisfactory determination of tooth shade.
Currently, most intraoral cameras include a sheath to cover the illuminating portion and/or image sensor. Conventional sheaths are disposable, so that they may be replaced if they accidentally or intentionally come in contact with a patient's mouth. By replacing a sheath between measurements on different patients, a dentist may prevent spread of contaminants, such as infectious agents, from a first patient to subsequent patient. Although these protective sheaths prevent spread of contaminants, their functionality is limited exclusively to this sanitary purpose.
Conventional intraoral cameras also include a handheld probe that a dentist inserts into a patient's mouth and collects color images with. Via a cable, the probe transmits collected color measurement to a computer that subsequently processes the measurements to create images and displays those images on a monitor for the dentist to view. The drawback in collecting images of a tooth with these conventional probes is that the dentist must look back and forth from the probe to the monitor to insure the probe is positioned over the tooth to obtain the desired image on the monitor. This, of course can cause unneeded frustration in aligning the probe to collect measurements of the tooth.
In many instances, intraoral cameras or parts thereof intentionally or accidentally come into contact with a patient's intraoral cavity thereby transmitting contaminants including infectious agents, saliva and/or food debris to the device. In addition to using sanitary sheaths as discussed above, operators of prior art intraoral cameras frequently clean or sterilize the cameras. This is often a tedious task, as the cameras include a plurality of buttons that are difficult to clean around and/or fiber optic bundles that are nearly impossible to sterilize without damaging the optical characteristics of the fibers because sterilization agents enter the fiber optics and degrade illumination or sensing capabilities. Accordingly, prior art camera users must exercise time-consuming care in operating and cleaning these cameras.
Typically, a dentist makes a shade determination visually using shade tabs. A prescription describing the restoration location and shade is sent to the dental laboratory. There, a technician attempts to duplicate the tooth shade to make prosthesis from available ceramic or synthetic materials. Once the prosthesis is manufactured, it is sent back to the dentist for installation in the patient.
Only after placing the prosthesis in proximity to the patient's damaged tooth and/or surrounding teeth can the dentist determine if the prosthesis is an acceptable duplicate of the damaged tooth. Of course, if the prosthesis does not match properly, the dentist must have a second prosthesis made by a lab incorporating his suggested modifications. A second shade determination of the tooth may even be required. The second prosthesis must also be compared by the dentist to the damaged tooth to insure a proper match. This process is very costly if multiple prosthetic replacements must be produced to create a satisfactory match. Moreover, this process consumes the time of patients who may come in for repeated visits before a matching prosthesis is created.
The aforementioned problems are overcome by the present invention wherein an optical measurement instrument uses searchlight illumination and calorimetric imaging to collect optical characteristic measurements of objects such as teeth.
The present invention uses searchlight illumination to uniformly illuminate an object when measuring the optical characteristics, such as color, shade, translucence, shape and/or gloss of that object. "Searchlight illumination" refers to illumination wherein all points of the object measured are illuminated with constant irradiance. Searchlight illumination is readily understood with reference to
Searchlight illumination offers significant benefits in the optical measurement context. First, image sensors used with searchlight illumination consistently collect optical information from light reflected from an object because the illumination is uniform over the object. Second, searchlight illumination does not vary significantly when the distance between the object and the instrument is increased as with conventional intraoral camera illumination systems. Accordingly, the positioning of the illumination source from the tooth may be varied without significantly sacrificing consistent optical information sensing. Third, with searchlight illumination, the occurrence of glare artifacts is diminished; and the optical characteristics of the object measured by a sensor are not washed out by this glare.
In another aspect of the invention, the preferred optical measurement instrument implements colorimetric imaging to measure the optical characteristics of objects. Colorimetric imaging is a process wherein multiple time-separated measurements, or "frames," of specific ranges of bandwidths of light in the visible spectrum are collected for all points on an object measured with an image sensor, preferably a monochrome image sensor. These frames are then combined to form a color image of the measured object, incorporating each collected frame and consequently each collected bandwidth. When combined, every point on each frame is aligned with its corresponding points on each other frame. When all the frames are combined, the resultant image depicts every point on the tooth and every bandwidth collected for that point on the tooth.
In the preferred embodiment, light reflected from an object, also referred to herein as "radiant flux," is sequentially transmitted through multiple filters that selectively absorb some wavelengths and transmit others to a monochromatic image sensor. These filters preferably have well controlled bandpass functions, that is, the filters allow specifically desired wavelengths of light to be transmitted to the sensor. These filters are specified by spectral transmittance curves. Preferably, the filters significantly attenuate wavelengths outside of the specified band pass function. Accordingly, these filters have very high color fidelity. Of course, depending on the application, the attenuation and transmission of certain wavelengths may be varied as desired.
In the preferred embodiment, each of these filters transmits a preferred range of bandwidths corresponding to Commission Internationale dei'Eclairage ("CIE") tristimulus value bandwidths, for example, X, Y, Z and X′ tristimulus bandwidths. Tristimulus values relate to the spectral sensitivity of the human eye. Instruments that measure using these bandwidth shapes correlate highly with data generated from human color perception. In this manner, the monochromatic image sensor collects only one bandwidth or range of bandwidths at time. These individual, collected bandwidths are recorded as frames by the image sensor, converted to electric signals and transferred to a processor. The processor combines and aligns these frames to form an image including all the collected bandwidths for every point on the image.
The problems inherent in the RGB sensors of prior art intraoral cameras, specifically pixel spacing separation and poor color fidelity, are eliminated with colorimetric imaging. Multiple bandwidths and the colors associated therewith are collected for each point on the tooth, rather than a single red, or green, or blue bandwidths. Additionally, because calorimetric imaging uses monochrome sensors having non-colored pixels, no defects attributable to poor color fidelity affect the collection of bandwidths. The resultant image created with colorimetric imaging is an extremely accurate representation of the true colors of the tooth to be replaced. Moreover, dental prosthesis created from this color data is much more likely to match the color of the replaced tooth.
In another aspect of the invention, the optical measuring instrument includes a multi-functional sanitary shield. In addition to preventing contamination from spreading from one patient to the next, these shields are of a predetermined length to establish an optimal distance between (a) the illumination source and/or (b) the image sensor, and the object measured. The shield also includes a reference strip, preferably off-white, that is included in images collected by the optical measurement instrument. The value of measurement associated with this reference strip in collected images is compared during measurement to values acquired during instrument calibration. This comparison provides a method of determining not only the lamp intensity variation—and the illumination in general—but also provides a method to determine changes in lamp color temperature. Both of these values are used to provide accurate color measurement data from the imaged tooth. In a preferred embodiment, the disposable shield is also constructed from an opaque material that prevents any ambient light from entering the shield and affecting any measurements made by the optical measurement instrument.
In another aspect of the invention, the optical measurement instrument is configured so that an operator of the instrument may view an image of an object on a display, preferably a liquid crystal display (LCD), along a line of sight that is the same as the line of sight along which an image sensor of the instrument collects an image of the object. Accordingly, the operator views the object from the same perspective as the image sensor collects the image of the object. With this "line of sight" viewing, the operator may align the image sensor of the optical measurement instrument so that it collects the exact image she desires simply by looking at the LCD. She need not look at the object she desires to measure and then turn her head to view a separate monitor to confirm that what she expected the sensor to collect is what is actually collected.
In another aspect of the invention the optical measurement instrument includes a sealed housing that facilitates sanitation and cleanup. The housing is sealed at all joints of adjoining panels that make up the housing. An LCD display is sealably incorporated into the housing. Preferably the display is touch sensitive so that buttons and their corresponding apertures in the housing body are eliminated. The housing body also includes an aperture through which optical measurement illumination and sensing is performed. This aperture is covered with a transparent window that is sealably mounted to the housing. Because the housing and this aperture are sealed from the environment, the sensitive illumination and sensing instrumentation therein is not subject to pollutants such as dust or sanitizing or cleaning agents and the like.
In another aspect of the invention, the optical characteristics of a manufactured prosthetic restoration is compared to the optical characteristics of the damaged tooth that the restoration is to replace before the restoration is shipped to a dentist for installation in a patient's mouth. In the preferred embodiment, a dentist collects an image of a damaged tooth or remaining surrounding teeth with an optical measurement instrument. This image is electronically forwarded to a dental prosthesis manufacturer who subsequently creates a prosthesis, or restoration, to replace the damaged tooth. Before shipping the prosthesis to the dentist, however, the manufacturer uses her own optical measurement instrument to collect an image of the prosthesis, and compares this image to the original image of the tooth. During comparison, the manufacturer insures that the optical characteristics of the image of the prosthesis accurately duplicate the optical characteristics of the image of the tooth. If characteristics are satisfactory, the manufacturer ships the prosthesis to the dentist for installation in the patient's mouth. If the characteristics of the prosthesis do not satisfactorily match the damaged tooth or surrounding teeth, a new prosthesis is made, and its image is collected and compared to the image of the damaged tooth or surrounding teeth. This process may be repeated until a satisfactory prosthesis is created.
By insuring that the optical characteristics of a prosthetic restoration accurately match the optical characteristics of the damaged tooth, dentists and patients can be assured that when the prosthesis arrives from the manufacturer, it will be a satisfactory match of the damaged tooth and acceptable for installation in the patient's mouth.
In the discussion herein, reference is made to an "object," "material," "surface," etc., and it should be understood that in general such a discussion may include teeth, dentures, gums, other prosthesis or restorations, dental cleaning material, dental adhesives or the like or other dental objects as well as any other objects or materials as the "object," "material," "surface," etc.
These and other objects, advantages and features of the invention will be more readily understood and appreciated by reference to the detailed description of the preferred embodiments and the drawings.
With reference to
The display 18 and the processor 20 may be separate or integrated as a unit as depicted. The display 18 is preferably a liquid crystal display (LCD). The LCD preferably has a touch screen interface to provide image control, data display and targeting feedback through a video viewfinder. As will be appreciated, any other display screens may be used. Alternatively, the optical measuring instrument may be connected via a cable (not shown) to a monitor or display (not shown) that is separate from the instrument for displaying images collected by the instrument.
In the preferred embodiment, the processor 20 is in electrical communication with the display 18, illumination assembly 80, and imaging subsystem 50. This processor is capable of processing digitized data collected from the imaging subsystem 50 and formatting it so that an image of that digitized data is output to the display screen 18. The processor preferably formats digitized measurements such as tristimulus value bandwidths collected by the image sensor 56 to form an image of the object measured on the display 18.
The processor 20 includes port 22 to connect to the instrument 10 to docking station, described in further detail below, to download images and/or data collected by the optical measuring instrument to a computer connected to the docking station for further analysis. Port 22 is also in electrical communication (not shown) with power source 90 so that the power source may be recharged when the instrument 10 is in its docking station.
Illuminator 80 is preferably mounted in the housing 12 in fixed relation to the imaging subsystem 50. This is accomplished via connectors 82 that may be of any configuration to hold the two assemblies in fixed relation. The fixed relation is preferably configured so that the illuminator 80 illuminates an object, such as a tooth, with light at a selected angle and the light reflected from the tooth is collected by the imaging subsystem 50 at a selected angle. In a preferred embodiment, the illuminator illuminates an object in an angle 18 degrees off normal and the imaging subsystem 50 collects light reflected from the object at an orientation normal to the object's surface. This configuration helps to reduce glare artifacts.
Of course, the illuminator 80 and imaging subsystem 50 may be configured in any angular configuration depending on the desired application. For example, the illumination and image capturing may be done both normal to the tooth. As will be also appreciated, the relation between the illuminator assembly 80 and imaging subsystem 50 may be configured such that with a combination of a beam splitter as known in the art, illumination and captuir of light reflected from an object may be done normal to an object's surface.
As will be appreciated, other conventional illumination assemblies may be substituted in the optical measurement instrument as the application requires. The assemblies may also include polarizers that limit the effect of specular gloss in the captured image.
The preferred embodiment of the optical measuring instrument uses searchlight illumination to illuminate objects during optical characteristic measurements. As used herein, "optical characteristics" means characteristics such as color, shade, translucence, gloss and/or shape. "Searchlight illumination" means illumination wherein the object measured is illuminated with constant irradiance. This definition is more readily understood with reference to
As explained,
At any point z along the optical axis where z<zc, the diameter of the uniform field �F can be approximated by:
The region of uniform irradiance can be extended in length along the optical axis 745 as illustrated in
The advantage of searchlight illumination over conventional illumination techniques is illustrated in the graph of
As shown, the intensity along the central axis from the theoretical point source 706 and fiber optic illuminator 704 is very strong when the target is near these sources; but that intensity rapidly decreases as the relative distance from the source to the target increases. In contrast, with a searchlight illuminator 702, which, by definition, has constant irradiance, the intensity along the optical axis remains substantially uniform at a distance within the operating range of the illuminator, as depicted by example here, from a relative distance of about 0.95 to about 1.0. At a distance somewhat greater than 1.05, the intensity from the searchlight gradually begins to decrease; but at a rate much less than the theoretical point source 706 and the conventional fiber optic illuminator 704.
Of course, as the illumination source distance to target is significantly increased (not shown on graph), even the searchlight illumination intensity will begin to decrease along the optical axis. But, for purposes of the carrying out the preferred embodiment of the present invention, illuminated targets or objects are positioned at a pre-selected distance from the searchlight source so that they are substantially within the region of constant irradiance.
The graph of
The optical measuring instrument of the preferred embodiment uses searchlight illumination to illuminate an object while the object's optical characteristics are measured.
With reference to
The searchlight illuminator 80, including source 518 and lens 528, preferably is placed a pre-selected distance D from the center of the object for which optical measurements are to be collected. The center of the nominal object or target may be placed at about 50 millimeters to about 100 millimeters from the lens, preferably from about 60 millimeters to about 70 millimeters from the lens, more preferably from about 63 millimeters to about 67 millimeters from the lens, and most preferably, about 65 millimeters from the lens. This distance D establishes a reference distance within which all points of the object optically measured are illuminated in the region of constant irradiance 560. For purposes of taking optical measurements of teeth, it is preferable to illuminate a substantial portion of the tooth or remaining teeth with the region of constant irradiance 560.
To establish distance D and insure that the points of the object measured are within the region of constant irradiance, a spacer is used to separate the illuminator 80 from the tooth. Preferably, a sanitary shield, described in further detail below, is attached to the optical measuring instrument that includes illuminator so that when the shield is disposed against or adjacent to the tooth, the distance D is established and the tooth is positioned in the region of constant irradiance. As will be appreciated, the illuminator 80 is positioned so that the light of the region of constant irradiance is reflected and detected by the imaging subsystem of the preferred optical measuring instrument described in further detail below.
With reference to
Light rays from light source 818 are projected through aperture 820 whose size is determined by the desired application and distribution of searchlight illumination by the illuminator 80. The aperture optionally may be covered by heat absorber 814 which is preferably in the form of a heat absorbing glass plate or synthetic material. The heat absorber 814 absorbs excessive heat created by the light source 818. Of course, this heat absorber 814 may be omitted in applications where there is no concern of heat buildup.
Illuminator 80 also includes a light limiting stop 832 that precisely configures the light from the aperture 820 on the lens 828. As will be appreciated, optical back scatter fins 852 are included in the housing 850 to prevent excessive back scatter of light that may confound the light transmitted by the light source 818 to the lens 828. Of course, the fins 852 may be colored, such as with a black or dark color to further reduce back scatter of light.
The illuminator housing holds lens 828 a predetermined distance from light source 818 to optimize the searchlight illumination projected from the illuminator 80. Of course, as discussed above in reference to
With reference to
With reference to
With particular reference to
Alternatively, the imaging subsystem may be configured so that the filter assembly is positioned between an illumination source and the object measured (not shown). In this manner, light from the illumination source would be transmitted through the filter assembly elements before being reflected from the tooth; however, reflected light impinging on the image sensor would still be of bandwidths selectively transmitted by the filter elements.
The lens 58 preferably has low chromatic aberration over the visible light spectrum from the 380 nm to 700 nm wavelength range. The lens focuses light L toward the image sensor 56, and causes the light to be transmitted through the elements of the filter wheel 60 a–60 f in the process. The filter wheel assembly of the preferred embodiment is a sector of 180 degrees including six elements. Of course, the assembly may be of any shape and include any number of filter elements.
The filter wheel assembly 60 of the preferred embodiment includes filter elements 60 a–f, wherein filter elements 60 a–d are of pre-selected band pass functions. "Band pass function" means information that is used to specify how a filter absorbs specific wavelengths of light as that light, also referred to as "radiant flux," is transmitted through a material. More preferably, filter element 60 a has a band pass function that permits it to transmit only X tristimulus value bandwidths and attenuate all other bandwidths; 60 b has a band pass function that permits it to transmit only Y tristimulus value bandwidths and attenuate all other bandwidths; 60 c has a band pass function that permits it to transmit only Z tristimulus value bandwidths and attenuate all other bandwidths; and 60 d has a band pass function that permits it to transmit only X′ tristimulus value bandwidths and attenuate all other bandwidths. These filters consistently attenuate bandwidths outside selected bandwidths to less than about 1/40th, preferably less than about 1/100th, and more preferably less than about 1/1000th of the value of the maximum transmittance of the filter. Of course, the filter elements 60 a–d may have any band pass function and attenuation as desired, and they may be altered in number so that only a selected number of filters are used in measurement.
Optionally, the filter assembly 60 may include opaque element 60 e to establish dark current information of the image sensor 56. Dark current information is current that flows in an image sensor when no optical radiation is impinging on the sensor. This current effectively distorts the electronic signals transmitted by the sensor to the processor. Accordingly, it is preferred to measure this dark current information and subtract it from the electronic signals generated during collection of bandwidths transmitted to the sensor so that subsequent optical characteristic measurements do not include this dark current information. The filter wheel may also include an open element space 60 f that transmits all light wavelengths to the image sensor. Transmitting all light wavelengths to the image sensor may be desired when initially acquiring an image of an object to help identify regions of the tooth that have high gloss.
With reference to
The stepped motor 54 is mounted to the back of the support plate 66 in a manner that limits contamination, magnetic field interaction, and heat transfer from the motor to the image sensor 56. The stepper motor preferably rotates the sectored filter wheel 60 in an indexed versus free-spinning manner.
The image sensor 56 is preferably a complimentary metal-oxide semiconductor (CMOS). As will be appreciated, any monochromatic sensor or photo detector may be substituted for the CMOS, including but not limited to a charged coupling instrument (CCD) sensor. As will be appreciated, the image sensor collects or captures bandwidths of the light L that is transmitted through respective filter elements 60 a–d, converts those functions to digitized form, and transfers the digitized form, also referred to as electronic signals, to the processor 20.
The stepper motor 54 and image sensor 56 are all synchronized, preferably by the processor 20, so that the image sensor 56 collects the bandwidths transmitted through each filter element 60 a–d when those filters are aligned one by one over the image sensor 56. The position sensor provides feedback via interaction with index 69 to the processor 20 to initialize and deactivate the stepper motor 54 in a desired manner.
With reference to
The alignment of elements 60 a–f is controlled by stepper motor 54 which is controlled by processor 20. In the preferred embodiment, when color measurement of light L reflected from a tooth is initialized, the stepper motor is in a "park" mode; that is, index 69 is aligned with position sensor 62. During measurement, the processor directs the stepper motor to rotate from the park mode through a plurality of partial movements consequently turning the filter wheel assembly 60 a plurality of pre-selected angles. These angles are such that each filter wheel element 60 a–60 f is positioned over the image sensor so that image sensor 56 collects a frame of data for light transmitted through each of the filter elements individually or for dark current into motion when element 60 e is positioned over the sensor. In this manner, only one bandwidth is transmitted to and captured by the image sensor at a given time or in a single frame.
In the preferred embodiment, the image sensor 56 takes three color measurements of a tooth. Each measurement comprises nine frames of data, which are subsequently stored in a processor 20 and combined to form a single measurement, or "image" of the tooth. Those frames represent two transmissions of X bandwidths of light L through filter 60 a two transmissions of Y bandwidths of light L through filter 60 b two transmissions of Z bandwidths of light L through filter 60 c, two transmissions of X′ bandwidths of light L through filter 60 d, and a single dark current information frame when the opaque element 60 e is positioned over the image sensor 56. This duplication of frames helps to integrate over time the collected bandwidths and may provide data needed for stabilization of the image.
As used herein, "stabilization" means combining the frames of data collected at different points in time so that the resultant image does not indicate that the optical measuring instrument was moved between the points in time during which the frames were taken. Multiple frames reflected by the sensor are separated in time because it takes a small fraction of time to collect a first frame, through, for example, filter element 60 a, move the filter wheel assembly 60 with the stepper motor 54 and collect the next frame through, for example, filter element 60 b. During this small fraction of time, the operator of the optical measurement instrument may accidentally move the instrument by rotating or shaking it. Accordingly, the frame collected for one filter element may be slightly different from other frames and the frames will not match up point for point. To correct this, once the data is converted to digitized form by the sensor and communicated to the processor, the processor uses special algorithms to align the frames so that a substantial number of points on one frame collected match up with a substantial number of points on the other frames collected.
A flow diagram conceptually outlining the process of aligning frames is depicted in
If no is the response to query 1008, the process proceeds with step 1010, where an error function that assesses column misalignment of the two images is calculated. In step 1012, one image is repositioned to remove column misalignment. In step 1014, an error function that assesses row misalignment is calculated. In step 1016, correlation is assessed; if correlation has degraded, then row movement is undone. In step 1018, one image is repositioned to remove row misalignment. Correlation is assessed in 1020; if it has degraded, then column movement is undone. An error function is calculated that assesses angular misalignment in step 1024. In step 1026, correlation is assessed. If correlation has degraded, then the rotation is undone.
In step 1028, a query is offered; has the loop been met? If yes, the process skips to macro step 2, and 1030, where it returns to final correlation value loop count, loop count, row offset, column offset and angular rotation values. The process proceeds to being done 1032. In step 1028, if the response is no, a inter-image correlation value is computed 1034; and the process skips to macro step 1 again to repeat all steps as necessary any number of times so that the process proceeds to done 1032. Of course, the preferred process may be modified in sequence. Steps may be modified and/or selectively repeated. Different steps may be added as well, depending on the desired application.
When the points of each frame are aligned with all the points of all other frames, the frames are collectively displayed to form an image of the object that reflected the light collected by the sensor. Preferably, a substantial number, if not all, of the points of this image will include all the tristimulus value bandwidths, that is the X, Y, Z and X′ tristimulus bandwidths that were collected by the image sensor. This image is preferably displayed on display 18 and stored in microprocessor 20. The image may be downloaded from the microprocessor 20 to a personal computer. In embodiments where only tristimulus bandwidths are collected by the imaging system, it will be appreciated that there is no need to perform lengthy calculations to derive tristimulus values if the output of the system be in tristimulus value format.
As will be further appreciated, the bandwidths collected by the image sensor may be combined and averaged in an image of the object into regions of uniform bandwidths. It is also possible to arithmetically combine into color zones those adjacent image points where color deviation between these adjacent image points do not exceed a predetermined value. In this way, measured tooth color can be subdivided into several color zones having different colors or bandwidths. The maximum number of such color zones in a prosthesis may be limited because a dentist or restoration manufacturer usually subdivides a prosthesis only into a limited number of color zones.
In addition to matching points to stabilize and/or align the frames to form an image of the tooth, the processor may also detect measurement errors if frames have been improperly collected. For example, if the operator of the optical measurement instrument drastically rotates or moves up and down or side to side the measuring instrument while sequential frames are being collected, one or more of the collected frames may be drastically different from the other. For example, one frame may be of a tooth and the next may be of a gum due to the drastic movement of the optical measuring instrument between frames. Accordingly, it would be difficult to align the points of one frame with the corresponding points of other frames because the frames would be quite different.
In cases where the processor detects that the frames collected are sufficiently different from one another so that corresponding points of different frames cannot be matched to form an image of the tooth, the processor indicates to the operator that the measurement must be redone. This indication may be communicated via display means or any other conventional alarm means. Accordingly, the operator retakes the optical characteristic measurement of the tooth to collect satisfactory data. As a result, the optical measuring instrument insures accurate and complete images are collected for further processing and manufacture of dental prosthesis.
Additionally, the processor preferably has the capability to store three or more images of object. These images may be recombined or welded together using appropriate welding software to combine multiple images into a single image. For example, dentists may use welding software to arrange individual images downloaded from the optical measuring instrument into a single image that replicates the configuration of teeth surrounding a damaged tooth in a patient's mouth.
The shield used in conjunction with the preferred optical measurement instrument is generally depicted in
In operation, the reference strip is included in the image field 350 when the optical measurement instrument collects an image of an object. The measured reference strip values are compared to values acquired during instrument calibration. This comparison provides a method of determining not only the lamp intensity variation, but also provides a method to determine changes in lamp color temperature. Both of these values must be known to provide accurate color measurement data from the imaged tooth. Lamp compensation factors are applied to all other objects in the image field 350, such as a tooth to determine the true colors of those objects. A true color image of the object may then be produced from these true colors.
The interior of the shield's hollow body is preferably opaque or otherwise colored with a dark material that prevents ambient light on the exterior of the shield from entering into the shield and polluting the data collected in the image field 350.
The shield 300 may also include indicia 340 disposed on first end 310. The indicia are preferably configured on end 310 so that it is included in the image field 350 when an image is collected. These indicia may be of any type, but preferably indicates the origin, such as manufacturer or distributor of the disposal shield to prevent counterfeiting thereof. The indicia 340 may also include patient information, a shield lot number, an expiration date, or any other information relevant to the patient or the optical measuring instrument. The indicia may be printed on, included in, affixed to or otherwise associated with the shield in any conventional manner. For example, the indicia may be a printed adhesive label or a barcode.
In the preferred embodiment, the shield establishes a predetermined distance to an object from an illumination source 80 or image sensor 56 as depicted in
The shield 300 may be secured to the optical measuring instrument as depicted in
As with appreciated, the shield may be of paper or plastic or other material which may be disposable, cleanable, reusable or the like in order to address any contamination concerns that may exist in a particular application. The shield may also be disposable or reusable. In the case of reusable shields, the shield is preferably constructed of material that can withstand sterilization in a typical autoclave, hot steam, chemiclave or sterilizing system.
Once the image sensor takes a measurement of the object in the line of sensing 402, that measurement is processed by the processor of the instrument (see
With the image displayed on the display 18, an operator 6 may view the display along a line of viewing 400. This line of viewing 400 is aligned with the line of sensing 402 so that the operator 6 views the tooth on the display in the same perspective as the image sensor senses the tooth. Manipulation of the line of sensing 402 preferably corresponds to a different image being output on the display 18. For example, when an operator moves the device, and consequently the line of sensing 402, to the right of the tooth 4, the image output on the screen 18 will correspond to whatever object is to the right of the tooth. More basically, a user may manipulate the device to realign the line of sensing by viewing an image on the screen without having to reverse or otherwise alter his normal thought process for acquiring and viewing an image.
With reference to
The optical measurement instrument 10 is configured in any way that allows the operator to manipulate the instrument 10 and simultaneously view the same image that the sensor senses on a display on the instrument without periodically having to look away from the display and realign the image sensor's line of sensing. Accordingly, the operator may view the display alone along line of viewing 400 to precisely align the line of sensing 402 so that the instrument collects the image of the tooth as desired.
With reference to
The aperture is circumferentiated by an internal lip 210 that is preferably formed as part of the housing 12. Disposed over the lip is window or cover panel 230. Preferably, this cover is made from plastic, glass or other synthetic material that allows high efficiency transmission of light through it. Disposed between the lip 210 and the window 230 is seal 220. The seal may be any gasket or seal, for example a sealing adhesive, that prevents "pollutants"—meaning dust, dirt, debris, moisture, cleaning agents, and chemicals—from entering the interior of the housing body 12 through or around the aperture 240.
The display 18 is preferably sealed to or into the housing 12 in a manner that prevents pollutants from entering the interior as well. It is preferred that the display is touch sensitive and is able to provide a means to control and operate the device. In this manner, no difficult to clean around external buttons are included in the housing.
Subparts 12 a and 12 b are preferably seated together in a manner that also prevents pollutants from entering the interior of the housing body along the portions of the subparts where the subparts connect to or seat against one another.
As explained, all of the above elements, the sealed window 240, the mated subparts 12 a, 12 b, and the display 18, prevent pollutants from entering the interior of the device when the pollutants come in contact with the device, such as when the device is wiped down with cleaning or sterilizing agents, or when the device is dropped on a dirty floor. However, these elements do not significantly prevent the pollutants from entering the interior of the device if the device is fully submersed in pollutants, for example, if the device is submersed in liquid cleaning agents.
The device of the present invention in some embodiments may include a port 22 or other connection for communication with a docking station (
The optical measurement instrument is easily sanitized and/or sterilized. Users may clean it by wiping it down without significant worry of leaking sanitizing or sterilizing agents or other cleaning agents into the interior of the housing 12, a consequence that potentially might damage the internal components of the instrument. Of course, care must be taken to prevent excessive exposure of a connection or port to pollutants to prevent those pollutants from entering the interior of the device. In periods of non-use, or use in dusty regions, the risk of dust or debris entering the housing is significantly decreased.
The preferred process of creating a dental restoration or prosthesis from optical measurements taken from a damaged tooth or surrounding teeth will now be described. To begin, a dentist uses the preferred optical measuring instrument to measure the optical characteristics of a tooth or teeth that surround an area that was previously occupied by a tooth. These optical measurements are converted to an image or plurality of images in the optical measurement instrument. The images may be downloaded from the optical measurement instrument to a computer where they may be stored. Of course, the image may be stored in any appropriate electronic file format. Once the image is stored in the computer, it forms what is referred to as a restoration file. From this restoration file, the measured optical characteristics may be mathematically manipulated by the computer to be viewed as an average characteristic map, as a grid of individual characteristics, as a contoured characteristic map, or any other desirable format as will be appreciated by those skilled in the art.
Next, the dentist transmits the restoration file to a restorative prosthesis-manufacturing laboratory with any acceptable means. Preferably, however, the file is forwarded using electronic network correspondence.
At the lab, a technician downloads the restoration file to reconstruct the patient's mouth, and in particular, the new prosthetic replacement for the damaged or missing tooth. Software capable of this reconstruction is available from X-Rite, Incorporated of Grandville, Mich. After the technician creates the restorative prosthesis, an image of the prosthesis, preferably with the optical measuring instrument of the preferred embodiment.
The inlet of the prosthesis is inserted into an image of patient's mouth derived from the restoration file to determine quality and accuracy of the restoration. This may be done in several ways. First, the technician may use her own optical measuring instrument to take measurements of the prosthesis to create an image of the prosthesis also referred to as "prosthesis data." She then takes this image and inserts it into an image of the patient's mouth taken from the restoration file. Of course, the technician may also compare the prosthesis data to the image of the original tooth, if one exits. The technician conducts a comparison of the image of the tooth to the image of the patient's mouth or damaged tooth before the restoration is shipped from the lab. The technician may then determine the quality and accuracy of the restoration and decide whether or not to ship it to the dentist for installation in the patient's mouth.
In a second alternative embodiment, the technician may use the optical measuring instrument to take measurements of the prosthetic restoration to create an image of the prosthesis and send that image to the dentist. The dentist may then visually insert the new tooth image into an existing image of the patient's mouth to determine the quality and accuracy of the restoration. Based on his or her own judgment, the dentist may then contact the lab to confirm or decline the restoration. In cases where the restoration is confirmed, the lab will ship the restoration to the dentist for installation in the patient's mouth. In cases where the dentist declines the restoration because it is not an acceptable match, the lab constructs another restoration and takes a new image of that restoration. The new image is forwarded to the dentist to compare that new image to the image of the damaged tooth. This process may be repeated until an accurate restoration is created.
In a third alternative embodiment, the technician may simply create the prosthesis and send it to the dentist. The dentist uses her own optical measurement instrument to obtain an image of the prosthesis. That prosthesis data is visually inserted into an image of the patient's mouth or compared to an image of the damaged tooth to determine the quality and accuracy of the restoration. If the restoration is acceptable, the dentist will install it in the patient's mouth. If the restoration is unacceptable, the dentist may request the lab to create another restoration or alter the restoration in a manner to make it an accurate duplicate of the original tooth for which it was designed to replace.
With reference to
In an alternative embodiment, the instrument may include a transmitter and/or receiver so that it can communicate with another instrument, with a docking station and/or directly with a computing device using a wireless connection wherein data may be transported through radio frequencies, light modulations, or other remote wireless communication means.
The above descriptions are those of the preferred embodiments of the invention. Various alterations and changes can be made without departing from the spirit and broader aspects of the invention as defined in the appended claims, which are to be interpreted in accordance with the principles of patent law including the doctrine of equivalents. Any references to claim elements in the singular, for example, using the articles "a," "an," "the," or "said," is not to be construed as limiting the element to the singular.
|Cited Patent
|Filing date
|Publication date
|8 Apr 1966
|1 Apr 1969
|Hans Adler
|Color and surface structure comparator
|3 Sep 1971
|11 Dec 1973
|Itek Corp
|System for analyzing multicolored scenes
|22 Feb 1972
|9 Apr 1974
|Dennis D
|Color shade analyzer
|18 Aug 1971
|30 Dec 1975
|Bates Harry E
|High speed optical wavelength detection system
|22 Aug 1974
|19 Oct 1976
|Weber Dental Mfg. Co., Div. Of Sterndent Corporation
|Tristimulus colorimeter for use in the fabrication of artificial teeth
|21 Apr 1976
|20 Jun 1978
|Sterndent Corporation
|Method of using a particular tristimulus colorimeter in making artificial teeth
|7 Sep 1976
|14 Nov 1978
|Sterndent Corporation
|Tristimulus colorimeter
|2 Nov 1976
|26 Dec 1978
|Sterndent Corporation
|Tristimulus colormeter with internal reflectance compensation
|10 Apr 1978
|30 Dec 1980
|Max Planck Gesellschaft Zur Forderung Der Wissenschaften
|Spectral photometer for medical use in determining skin coloration
|25 Jun 1979
|27 Jan 1981
|Canadian Instrumentation And Research
|Automatic computing color meter
|24 Nov 1981
|8 Nov 1983
|Dr.-Ing. Rudolf Hell Gmbh
|Method and circuit for recognition of colors
|15 Oct 1980
|21 May 1985
|Harmen Broersma
|Colour analyser
|7 May 1982
|15 Oct 1985
|Omron Tateisi Electronics Co.
|Color sensing device
|23 Aug 1984
|11 Mar 1986
|Moermann Werner H
|Method and apparatus for the fabrication of custom-shaped implants
|14 Apr 1983
|9 Sep 1986
|Francois Duret
|Apparatus for taking odontological or medical impressions
|12 Dec 1984
|18 Nov 1986
|Dr. Ing. Rudolf Hell Gmbh
|Method and apparatus for the recognition of hues and colors
|18 Feb 1984
|31 Mar 1987
|Colorgen, Inc.
|Methods for determining the proper coloring for a tooth replica
|22 Nov 1983
|7 Jul 1987
|Hitachi Maxell, Ltd.
|Color sensor
|27 Sep 1984
|8 Sep 1987
|E. I. Du Pont De Nemours And Company
|Process for matching color of paint to a colored surface
|20 Apr 1987
|20 Sep 1988
|Omron Tateisi Electronics Co.
|Image analyzing apparatus
|11 Aug 1986
|14 Mar 1989
|Megatronics, Incorporated
|Optical color analysis process
|9 May 1986
|4 Apr 1989
|Wilhelm Sedlbauer Gmbh
|Processing device for local irradiation with visible or invisible light
|12 Dec 1986
|6 Jun 1989
|Bertin & Cie
|Method and apparatus for determining color, in particular of a dental prosthesis
|5 Jun 1987
|6 Jun 1989
|Marco Brandestini
|Method and apparatus for the three-dimensional registration and display of prepared teeth
|16 Feb 1988
|21 Nov 1989
|Colorgen, Inc.
|Remote color measurement device
|7 Aug 1985
|23 Oct 1990
|Smith & Nephew Dyonics, Inc.
|Fiber bundle illumination system
|24 Aug 1989
|26 Mar 1991
|Ciba-Geigy Corporation
|Process and apparatus for the preparation of color formulations utilizing polarized light in spectrophotometry
|2 Feb 1990
|30 Apr 1991
|Colwell/General, Inc.
|Objective color notation system
|15 May 1989
|21 May 1991
|Jay Ascher
|Removable and disposable extension for a light guide of a dental curing light and its method of use
|18 Jan 1990
|24 Sep 1991
|Fuji Optical Systems, Inc.
|Dental instrument including laser device and electronic video dental camera
|20 Jul 1990
|23 Jun 1992
|New Image Industries, Inc.
|Modular view lens attachment for micro video imaging camera
|2 Jan 1990
|22 Sep 1992
|Megatronics, Inc.
|Method for correlating color measuring scales
|19 Sep 1990
|13 Oct 1992
|E. I. Du Pont De Nemours And Company
|Method and apparatus for analyzing the appearance features of a surface
|14 Jul 1989
|5 Jan 1993
|Garibaldi Pty Ltd.
|Computerized color matching
|16 Sep 1991
|27 Jul 1993
|Ppg Industries, Inc.
|Color matching and characterization of surface coatings
|5 Jan 1993
|31 Aug 1993
|Thompson Charles C
|Method for shade selection in restorative dentistry
|19 Feb 1992
|31 Aug 1993
|Itt Corporation
|Fiber optic imaging device and methods
|14 Dec 1992
|11 Jan 1994
|Hewlett-Packard Company
|Method and apparatus for providing illumination compensation in a document scanner
|22 Feb 1993
|17 May 1994
|Chromatics Color Sciences International Inc.
|Method and instrument for selecting personal compatible colors
|9 Jun 1993
|7 Jun 1994
|Kollmorgen Corporation
|Handheld portable spectrophotometer
|22 Feb 1994
|13 Dec 1994
|Krzyminski; Harald
|Hand-held instrument for measuring reflections on colored control fields of printed sheets
|15 Dec 1992
|17 Jan 1995
|Bertin & Cie
|Method and apparatus for determining the color of a translucent object such as a tooth
|5 Nov 1993
|25 Apr 1995
|British Technology Group Usa Inc.
|Endoscopic device for intraoral use
|20 Dec 1993
|27 Jun 1995
|Bertin & Cie
|Method and apparatus for determining the color of an object that is transparent, diffusing, and absorbent, such as a tooth, in particular
|5 Apr 1995
|19 Dec 1995
|Mcdonnell Douglas Corporation
|Digital image system and method for determining surface reflective and refractive characteristics of objects
|4 May 1994
|9 Jan 1996
|Killmorgen Corporation
|Spectrophotometer and radiometric measurement apparatus
|28 Jul 1993
|16 Jan 1996
|Siemens Aktiengesellschaft
|Method and apparatus for treating hard dental substances
|7 Jun 1993
|20 Feb 1996
|Centre De Recherche Industrielle Du Quebec
|Method and apparatus for sensing color of articles
|4 Oct 1993
|11 Jun 1996
|General Electric Company
|Imaging color sensor
|13 Sep 1994
|25 Jun 1996
|Moritex Corporation
|Cordless light source
|15 Mar 1995
|4 Feb 1997
|Erich M. N. Hibbard
|Autoclavable rigid endoscope
|1 Aug 1995
|3 Jun 1997
|Acer Peripherals, Inc.
|Automatic intensity control of the lamp within image scanning apparatus
|31 Jan 1996
|5 Aug 1997
|Hewlett-Packard Co.
|Method and apparatus for dynamic white point adjustment
|31 Mar 1995
|19 Aug 1997
|Medrx, Inc.
|Video otoscope and optical lens system therefor
|13 Aug 1993
|26 Aug 1997
|Siemens Aktiengesellschaft
|Video camera fashioned as a handpiece for observing subjects in mouth of a patient
|28 Jul 1995
|25 Nov 1997
|Dentalase Corporation
|Dental tooth color detector apparatus and method
|19 Apr 1995
|25 Nov 1997
|Hamlin; David
|Endoscopic device
|2 Jan 1996
|28 Apr 1998
|Lj Laboratories, L.L.C.
|Apparatus for determining optical characteristics of an object
|2 Jan 1996
|2 Jun 1998
|Lj Laboratories, L.L.C.
|Method for determing optical characteristics of teeth
|26 Jun 1995
|16 Jun 1998
|Murljacic; Maryann Lehmann
|Tooth shade analyzer system and methods
|2 Dec 1996
|25 Aug 1998
|Mht Optic Research Ag
|Method for determining the color stimulus specification of translucent objects and an apparatus for performing the method
|23 Dec 1996
|13 Oct 1998
|Nustek Systems, Inc.
|Method and apparatus for compensating illuminance error of a light source
|25 Jul 1997
|17 Nov 1998
|Dentsply International Inc.
|Portable dental camera system and method
|22 Sep 1995
|15 Dec 1998
|Color And Appearance Technology, Inc.
|Colorimetric imaging system for measuring color and appearance
|12 Aug 1997
|22 Dec 1998
|Lj Laboratories, L.L.C.
|Apparatus and method for measuring the color of teeth
|29 Nov 1994
|22 Dec 1998
|Nobel Biocare Ab
|Method and arrangement for collecting data for production of replacement dental parts for the human body
|12 Aug 1997
|16 Feb 1999
|Lj Laboratories, L.L.C.
|Apparatus and method for measuring the color of teeth
|1 Jul 1997
|9 Mar 1999
|L J Laboratories, L.L.C.
|Apparatus and method for measuring optical characteristics of teeth
|12 Aug 1997
|16 Mar 1999
|Lj Laboratories, L.L.C.
|Apparatus for measuring optical properties
|6 Feb 1997
|13 Apr 1999
|Welch Allyn, Inc.
|Gripping handle for diagnostic instrument
|21 Mar 1997
|11 May 1999
|Lifespex, Incorporated
|Method and apparatus for calibrating an optical probe
|3 Apr 1998
|20 Jul 1999
|Spectrx, Inc.
|Disposable calibration target
|20 May 1998
|5 Oct 1999
|Shade Analyzing Technologies, Inc.
|Tooth shade analyzer system and methods
|2 Jul 1997
|19 Oct 1999
|Dental Devices, Llc
|Dental color comparator device
|18 Jun 1997
|30 Nov 1999
|Hewlett-Packard Company
|Illumination system with white level calibration for hand-held scanner
|26 Sep 1996
|28 Dec 1999
|O'brien; William J.
|Tooth color matching system
|22 Dec 1998
|28 Dec 1999
|Deus Ex Machina Inc.
|Method and apparatus for determining the appearance of an object
|9 Jun 1998
|1 Feb 2000
|Integra Medical
|Camera with improved focus mechanism
|15 Oct 1998
|29 Feb 2000
|Jeneric/Pentron Incorporated
|Method for accurately preparing the color of a dental restoration
|2 Mar 1998
|29 Feb 2000
|Infrared Fiber Systems, Inc.
|Handheld infrared spectrometer
|8 Jan 1999
|14 Mar 2000
|Mht Optic Research
|Method and an apparatus for determining the color stimulus specification of an object
|12 Aug 1997
|21 Mar 2000
|Lj Laboratories, L.L.C.
|Apparatus and method for measuring color
|8 Sep 1998
|13 Jun 2000
|Olympus Optical Co., Ltd.
|Electronic camera
|23 Dec 1997
|25 Jul 2000
|Integra Medical
|Dental imaging system with digital motion video
|27 Jan 1999
|1 Aug 2000
|Welch Allyn, Inc.
|Gripping handle for diagnostic instrument
|30 Apr 1999
|15 Aug 2000
|Osterwalder; J. Martin
|Apparatus for curing resin in dentistry
|31 Mar 1998
|22 Aug 2000
|Welch Allyn, Inc.
|Compact imaging instrument system
|14 Aug 1997
|29 Aug 2000
|Rawicz; Andrew
|Method and apparatus for color matching of slightly colored translucent objects such as teeth and dental prosthesis, in particular
|9 Jul 1998
|12 Sep 2000
|Lj Laboratories, L.L.C.
|Apparatus and method for measuring optical characteristics of an object
|30 Aug 1999
|17 Oct 2000
|Shade Analyzing Technologies, Inc.
|Tooth shade analyzer system and methods
|1 Apr 1996
|13 Feb 2001
|Lj Laboratories, L.L.C.
|Picture frame with associated audio messages
|28 Oct 1999
|13 Feb 2001
|Lj Laboratories, L.L.C.
|Apparatus and method for measuring optical characteristics of an object
|6 May 1998
|20 Mar 2001
|Must Systems, Inc.
|Image scanner having brightness compensation function via output adjustment
|4 Oct 1999
|27 Mar 2001
|Shade Analyzing Technologies, Inc.
|System and methods for analyzing tooth shades
|3 Feb 2000
|3 Apr 2001
|Shade Analyzing Technologies, Inc.
|System and methods for analyzing tooth shades
|27 Jul 2000
|24 Apr 2001
|Lj Laboratories, Llc
|Apparatus and method for measuring optical characteristics of an object
|18 Sep 1998
|15 May 2001
|Agency Of Industrial Science And Technology
|Method for bleaching discolored tooth by titanium dioxide photocatalyst
|2 Jan 1997
|15 May 2001
|Lj Laboratories, L.L.C.
|Apparatus and method for measuring optical characteristics of an object
|23 Nov 1998
|7 Dec 1999
|Hamlin; David
|Endoscopic device
|General Characteristics of an LSD(R), (visited Oct. 26, 2000) <http: //www.poc.com/hgram/techdata/general.htm> Printout.
|International Searching Authority, Partial International Search in PCT Application PCT/US00/33478 issued on May 8, 2001.
|J.D. Scheuch, Modeling of constant irradiance illumination system, SPIE Journal, vol. 3428, at 22-27 (1998).
|US 6,100,988, 08/2000, Jung et al. (withdrawn)
|Citing Patent
|Filing date
|Publication date
|19 Feb 2010
|9 Sep 2010
|Maev Roman Gr
|Ultrasonic device for assessment of internal tooth structure
|29 May 2014
|12 May 2015
|Symbol Technologies, Inc.
|Mobile computer
|U.S. Classification
|356/407, 356/408, 433/29
|International Classification
|Cooperative Classification
|European Classification
|7 Mar 2001
Owner name: X-RITE INCORPORATED, MICHIGAN
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:OVERBECK, JAMES L.;VAN ANDEL, RICHARD J;GALEN, MICHAEL A;AND OTHERS;REEL/FRAME:011570/0915
Effective date: 20010213
|31 Jan 2006
Owner name: GOLDMAN SACHS CREDIT PARTNERS L.P., NEW YORK
Free format text: PATENT SECURITY AGREEMENT (SECOND LIEN);ASSIGNOR:X-RITE, INCORPORATED;REEL/FRAME:017097/0337
Effective date: 20060131
Owner name: GOLDMAN SACHS CREDIT PARTNERS L.P., NEW YORK
Free format text: PATENT SECURITY AGREEMENT (FIRST LIEN);ASSIGNOR:X-RITE, INCORPORATED;REEL/FRAME:017097/0263
Effective date: 20060131
|5 Nov 2007
Owner name: FIFTH THIRD BANK, A MICHIGAN BANKING CORPORATION,
Free format text: SECURITY AGREEMENT;ASSIGNORS:X-RITE, INCORPORATED;OTP, INCORPORATED;MONACO ACQUISITION COMPANY;AND OTHERS;REEL/FRAME:020064/0313
Effective date: 20071024
|8 Nov 2007
Owner name: THE BANK OF NEW YORK, AS COLLATERAL AGENT, NEW YOR
Free format text: PATENT SECURITY AGREEMENT (SECOND LIEN);ASSIGNOR:X-RITE, INCORPORATED;REEL/FRAME:020156/0569
Effective date: 20071024
|24 Apr 2008
Owner name: X-RITE INCORPORATED, MICHIGAN
Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:GOLDMAN SACHS CREDIT PARTNERS L.P., AS COLLATERAL AGENT;REEL/FRAME:020845/0285
Effective date: 20071024
|3 Jun 2008
Owner name: X-RITE, INCORPORATED, MICHIGAN
Free format text: PATENT RELEASE AND REASSIGNMENT FOR THE PATENT SECURITY AGREEMENT ORIGINALLY RECORDED AT REEL 017097, FRAME 0263 AND ASSIGNED AT REEL 018171, FRAME 0167;ASSIGNORS:FIFTH THIRD BANK;GOLDMAN SACHS CREDIT PARTNERS L.P.;REEL/FRAME:021029/0554
Effective date: 20071024
|29 Sep 2009
|Fee payment
Year of fee payment: 4
|6 Apr 2011
Owner name: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT, TE
Free format text: NOTICE OF GRANT OF SECURITY INTEREST IN PATENTS;ASSIGNOR:X-RITE, INCORPORATED;REEL/FRAME:026082/0713
Effective date: 20110330
|14 Apr 2011
Owner name: X-RITE GLOBAL, INCORPORATED, MICHIGAN
Free format text: PATENT RELEASE AND REASSIGNMENT;ASSIGNOR:FIFTH THIRD BANK, AN OHIO BANKING CORPORATION, AND SUCCESSOR BY MERGER TO FIFTH THIRD BANK, A MICHIGAN BANKING CORPORATION, AS COLLATERAL AGENT;REEL/FRAME:026126/0496
Effective date: 20110330
Owner name: OTP, INCORPORATED, MICHIGAN
Free format text: PATENT RELEASE AND REASSIGNMENT;ASSIGNOR:FIFTH THIRD BANK, AN OHIO BANKING CORPORATION, AND SUCCESSOR BY MERGER TO FIFTH THIRD BANK, A MICHIGAN BANKING CORPORATION, AS COLLATERAL AGENT;REEL/FRAME:026126/0496
Effective date: 20110330
Owner name: X-RITE, INCORPORATED, MICHIGAN
Free format text: PATENT RELEASE AND REASSIGNMENT;ASSIGNOR:FIFTH THIRD BANK, AN OHIO BANKING CORPORATION, AND SUCCESSOR BY MERGER TO FIFTH THIRD BANK, A MICHIGAN BANKING CORPORATION, AS COLLATERAL AGENT;REEL/FRAME:026126/0496
Effective date: 20110330
Owner name: GRETAGMACBETH LLC, MICHIGAN
Free format text: PATENT RELEASE AND REASSIGNMENT;ASSIGNOR:FIFTH THIRD BANK, AN OHIO BANKING CORPORATION, AND SUCCESSOR BY MERGER TO FIFTH THIRD BANK, A MICHIGAN BANKING CORPORATION, AS COLLATERAL AGENT;REEL/FRAME:026126/0496
Effective date: 20110330
Owner name: X-RITE HOLDINGS, INC., MICHIGAN
Free format text: PATENT RELEASE AND REASSIGNMENT;ASSIGNOR:FIFTH THIRD BANK, AN OHIO BANKING CORPORATION, AND SUCCESSOR BY MERGER TO FIFTH THIRD BANK, A MICHIGAN BANKING CORPORATION, AS COLLATERAL AGENT;REEL/FRAME:026126/0496
Effective date: 20110330
Owner name: MONACO ACQUISITION COMPANY, MICHIGAN
Free format text: PATENT RELEASE AND REASSIGNMENT;ASSIGNOR:FIFTH THIRD BANK, AN OHIO BANKING CORPORATION, AND SUCCESSOR BY MERGER TO FIFTH THIRD BANK, A MICHIGAN BANKING CORPORATION, AS COLLATERAL AGENT;REEL/FRAME:026126/0496
Effective date: 20110330
Owner name: PANTONE, INC., NEW JERSEY
Free format text: PATENT RELEASE AND REASSIGNMENT;ASSIGNOR:FIFTH THIRD BANK, AN OHIO BANKING CORPORATION, AND SUCCESSOR BY MERGER TO FIFTH THIRD BANK, A MICHIGAN BANKING CORPORATION, AS COLLATERAL AGENT;REEL/FRAME:026126/0496
Effective date: 20110330
|19 Apr 2011
Owner name: X-RITE HOLDINGS, INC., MICHIGAN
Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:THE BANK OF NEW YORK MELLON, AS AGENT;REEL/FRAME:026149/0681
Effective date: 20101001
Owner name: X-RITE, INCORPORATED, MICHIGAN
Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:THE BANK OF NEW YORK MELLON, AS AGENT;REEL/FRAME:026149/0681
Effective date: 20101001
Owner name: X-RITE GLOBAL, INCORPORATED, MICHIGAN
Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:THE BANK OF NEW YORK MELLON, AS AGENT;REEL/FRAME:026149/0681
Effective date: 20101001
Owner name: MONACO ACQUISITION COMPANY, MICHIGAN
Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:THE BANK OF NEW YORK MELLON, AS AGENT;REEL/FRAME:026149/0681
Effective date: 20101001
Owner name: GRETAGMACBETH, LLC, MICHIGAN
Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:THE BANK OF NEW YORK MELLON, AS AGENT;REEL/FRAME:026149/0681
Effective date: 20101001
Owner name: OTP, INCORPORATED, MICHIGAN
Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:THE BANK OF NEW YORK MELLON, AS AGENT;REEL/FRAME:026149/0681
Effective date: 20101001
Owner name: PANTONE, INC., NEW JERSEY
Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:THE BANK OF NEW YORK MELLON, AS AGENT;REEL/FRAME:026149/0681
Effective date: 20101001
|16 May 2012
Owner name: X-RITE, INC., MICHIGAN
Free format text: TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS RECORDED ON REEL 026082 FRAME 0713;ASSIGNOR:BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT;REEL/FRAME:028218/0671
Effective date: 20120515
|15 Nov 2013
|Fee payment
Year of fee payment: 8
|1 Jan 2018
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.)


text:
The three suspects, whose names have not been released, are believed to be involved in three burglaries Wednesday night on the Pottsville side of Crow Mountain and two Monday and Tuesday north and east of Atkins.
The suspects, who were reported to have been driving an all-white Ford Expedition, were located late Thursday on Crow Mountain.
"Myself and the chief deputy just happened to be driving around in the area and happened to drive up on them," DuVall said. "The chief deputy is the one who spotted the vehicle first, and he radioed myself and some other law enforcement and we proceeded to the area he was at."
DuVall said the PCSO partnered with Arkansas State Police, Atkins Police Department and an Arkansas constable in the arrest. He added items that had been stolen from different residences in the area were located when the arrests were made.
The burglaries are still under investigation and the PCSO is looking for other suspects, who may be in Conway or Faulkner counties, DuVall said.
The three suspects arrested Thursday will appear in felony bond hearings at Pope County District Court Monday.

subset: commoncrawl


text:
I just signed on to Kazaa and I'm havin some trouble figuring it out.
First, I put in my own E-mail address then later read I should have used a fake one. How do I change it to a fake one?
Secondly, I'm not sure if I am sharing the entire contents of my computer or just the folder that I chose. It looks like I'm sharing my entire harddrive.


text:
Coming to an ISP near you in July!
Just another reason to use SSL...
Coming to an ISP near you in July!
Just another reason to use SSL...
Who needs cloud storage when you got the NSA?
Torrentfreak reports they only track p2p downloads.
Usenet should be relatively safe, with or without SSL, for now.Those not engaging in file-sharing on P2P networks will probably notice very little (cyberlocker sharing is not covered), apart from ultimately having to help finance the scheme through their ISP bills.
I don't see my ISP on this list, perhaps I've been able to sidestep this issue? I wonder if there's a list of all ISPs involved in this.
Everything is brought to you by Fjohürs Lykkewe.
All the major ISP's are participating: Comcast, Time Warner, Verizon, Cablevision, et al.
http://news.cnet.com/8301-31001_3-57...ht-by-july-12/Last July, Comcast, Cablevision, Verizon, Time Warner Cable and other bandwidth providers announced that they had agreed to adopt policies designed to discourage customers from illegally downloading music, movies and software. Since then, the ISPs have been very quiet about their antipiracy measures.
Who needs cloud storage when you got the NSA?
It's time (or past time) for folks to start getting serious about VPN's.
i can not belive this all the stuff going on in the world and these F's are doing this what is wrong with this world
Last edited by sonic1; 03-15-2012 at 04:28 PM.
they have enough money already. its not about money at all.
Its all about power and control.
copyright shit is just a piss poor "excuse" to bring us 1984.
Who needs cloud storage when you got the NSA?


text:
Lewis cried foul, accusing McKeithen of wrongdoing in revealing her vote. But she's dead wrong. It is public information how elected officials vote, even for matters discussed in closed session. The action taken, and how each official voted on it, is 100% public information. (As an aside, Ms. Lewis must know this, since she didn't complain when Charles Marsala revealed how council members voted on the road impact fee in closed session).
Ms. Lewis surely recognized this compensation reduction was in the best interests of the tax paying residents she is supposed to represent when she cast her closed session vote.
Why then, was she expressing outrage at the resolution in public last night? Why was she demanding voting wait until her running mate, Cary Wiest, could join her to now vote against it?
What changed in the intervening time between the recent closed session vote and her behavior last night?
Answer: The APOA threw a fit over the resolution.
Peter Carpenter, do you still maintain Ms. Lewis has no allegiance to the APOA from their endorsement of her?
To me, this is proof positive they pull her strings.


text:
In just a few hours, we'll know the results,
When the last person casts a vote;
Between four white guys from three different cults,
And not lots of thought there to note.
The strangest of bedfellows, politics makes,
When they're racing, all hell bent for leather
Who cares what religion, they're all of them snakes
So they may as well all hang together
I see a silver lining in this. Yes, Catholicism (2 candidates), Mormonism (one) and Libertarianism (one) have each been termed "cults". Frankly, I don't much care whether they fit a technical definition, as it has always seemed clear to me that the usage simply implied that this group was A) bad, and B) not us. That's all.
But now, in conservatively religious South Carolina, the front-runners are members of religions that local pinheads have only recently demeaned as cults. And it doesn't seem to matter. So the silver lining is that the tribes of religion hold less power than they used to. The dark cloud, though, is that they may all agree that atheism, at least, is a cult.


text:
Eric Church
Photo: The Recording Academy
5 Questions With ... Eric Church
In August 2011 The Recording Academy sat down with chart-topping country artist Eric Church for an installment of its 5 Questions With … series. Following his recent nomination for the 54th Annual GRAMMY Awards, Church participated in another installment of 5 Questions With …. In an intimate interview setting, Church discussed being nominated for a GRAMMY, his song "Drink In My Hand" and reflected on his career since the release of his GRAMMY-nominated album, Chief.
"The Chief record changed everything in my career and my life," said Church. "Sometimes you feel like you're beating your head against the wall for six or seven years and it's like all of a sudden in the last six months, we're a six-year overnight success."
View video from 5 Questions With … Eric Church
Hailing from Granite Falls, N.C., Church began writing songs and taught himself how to play guitar at age 13. Upon graduating from college at Appalachian State University, Church moved to Nashville where he landed a publishing deal with Sony/ATV Tree and penned songs for other artists, including Terri Clark's "The World Needs A Drink," which he co-wrote with Casey Beathard. Church then met producer Jay Joyce and the two began recording demos, landing Church a record deal with Capitol Records Nashville. Church's debut album, 2006's Sinners Like Me, peaked at No. 29 on Billboard's Country Albums chart. Carolina followed in 2009, peaking at No. 17 on the Billboard 200 and spawning the gold-certified singles "Love Your Love The Most" and "Smoke A Little Smoke."
Released in July 2011, Chief debuted at No. 1 on the Billboard 200 and Country Albums charts, marking the first time a traditional country artist has garnered a No. 1 album on the Billboard 200 without having a previous No. 1 hit single since Tim McGraw's 1994 album, Not A Moment Too Soon. The album garnered Church his first GRAMMY nomination for Best Country Album.


text:
Well, actually, he became a double agent when he found out Lily was targeted, and he started teaching apparently about two months before the Potters were killed.Originally Posted by leahsm2
However, I agree that fresh out of Hogwarts Snape is a lot different than jaded spy who lost the love of his life Snape.
I think something you're not considering which bears STRONG consideration is this: based on his birthdate of January 1960, his first year at Hogwarts would have been September 1971. He would have left ("graduated") Hogwarts in June 1978. "Snape's Worst Memory", his fifth year, would have happened in June 1976, and Severus started teaching in September 1981, meaning that some of the students who saw Severus being routinely humiliated by MWPP would have then become Snape's students, and of those who didn't witness it, they'd have heard it from those other students who did see it, or their family members who did. I imagine it would have made teaching kind of difficult.
But you have to admit, both his opening Potions and his opening Defence speeches were pretty poetic. The guy's a serious academic and a bookwork that put Hermione Granger to shame.
Remember, too, he took the position at Hogwarts under the Dark Lord's orders. So he probably put a great deal of thought into that opening Potions speech, and by the time the tradition had been established, he'd thought a lot about his Defence speech, too.


text:
Winner of the Eco-Design and Sustainable Technology award at the Consumer Electronics Show, this is the illuminated Kindle 4 case that is powered by the sun. Unlike lesser lighted covers that draw power from the Kindle itself, this superior case has a solar panel built into its cover that powers the internal 1,500 mAh battery. The rechargeable battery powers the integrated LED light for up to 50 hours or recharges the Kindle to 80% of its maximum charge. The LED lamp extends from the case and casts bright, white light to illuminate the entire Kindle screen for more comfortable reading in low light. The case is made from durable leatherette, is only slightly larger the Kindle, and weighs only 1/2 lb., maintaining the Kindle's lightweight, easy-to-hold design. Kindle 4 Solar Case. 7 3/4" H x 4 1/2" W x 3/4" D. (8 1/2 oz.)
Lifetime Guarantee
The Solar Lighted Kindle 4 Case comes with The Hammacher Schlemmer Lifetime Guarantee. If this product ever disappoints you, for any reason, you may return your Hammacher Schlemmer purchase for exchange, credit, or refund.
Should you have any questions, we are available seven days a week (8:00 am – midnight EST). Please call 1-800-321-1484 to speak with a specialist or email us at firstname.lastname@example.com to receive a response within three business hours or less.
May we recommend:
COPYRIGHT © 2018 HAMMACHER SCHLEMMER & COMPANY, INC. ALL RIGHTS RESERVED. W109147 E. 57th ST • NEW YORK, NY • 10022 • 1-800-_321-1484


text:
COURSE: 10-112 Introduction to Human Communication (3)
An introduction to basic theories of communication, this course includes an examination of techniques of communication in three common communication settings: interpersonal relationships, group gatherings and public gatherings. The public context includes the presentation of a minimum of three speeches. At least one speech is an informative speech and at least one is persuasive. All students are required to give at least three speeches. Topics to be covered include nonverbal communication and listening.


text:
A somewhat unusual theft took place last week at Wal-Mart involving dishonesty in the self-check-out lane, according to a police report.
On March 14, at about 11:45 p.m., a call came in the Russellville Police Department claiming a man and woman were seen removing price tags off of inexpensive items and placing those tags onto similar, more expensive items.
But as if the criminal act was not enough for the couple, they would then bring the items back into the store to get refunds on the original prices of the items they "purchased."
Officer Blake Bradley discovered during questioning that the couple had tried to falsely exchange a set of satin sheets and a set of tools.
Although the prices of the two items were all the couple admitted to altering, a store manager verified it was not the couple's first attempt.
The two were taken to the Pope County Detention Center and issued citations for shoplifting.
Follow that beanie
Police have been looking for a 5-foot, 8-inch tall white male in his mid to early 20s with dark hair, weighing about 190 pounds who they believe attempted to steal a vehicle late last month.
At about 10 p.m., Officer Gary Collins was dispatched to Highway 7 North at the Holiday Inn when a man reported his truck had been stolen.
But by the time the officer began taking the report, a nearby female yelled, "There's your truck!" The truck she was referring to was about 100 yards away, also in the Holiday Inn parking lot.
Then, the report said, a man wearing blue jeans, a gray sweater with stripes, a blue coat and a black beanie got out of the truck, but when he realized a police car was parked where the victim's truck was taken from, he ran south across Lakefront Drive and into a wooded area.
The report stated a red "Under Armour" long-sleeve shirt and a red beanie were recovered next to the wooded area. But the black beanie with "RHS softball #9" spelled in red letters and a white Tommy Hilfiger coat were taken from the truck.
The suspect was not located, but the truck was turned back over to its owner.
How many cigarettes is that?
A local Exxon store manager reported last week the story was $2,523.34 short on its cigarette stock at inventory time.
At about 10:15 a.m., Officer Glenn Daniel met with the manager who told him she suspected one of the employees stole multiple cartons of cigarettes to sell, or exchange, them in return for drugs.
When asked, the manager told police of a drug problem one employee had and stated that employee admitted to "giving" a couple of cartons away.
The manager gave police three store surveillance video tapes and store inventory records for evidence, as well as a list of employees.


text:
The four Inland men accused of plotting to attack U.S. military bases in Afghanistan trained for months and discussed their eagerness to join al-Qaida and kill U.S. soldiers, federal officials said.
FBI agents arrested three of the men Friday, Nov. 16, the day they were to drive to Mexico City on the first leg of a journey to meet up with the fourth man in Afghanistan.
"They planned to travel to Afghanistan and engage in violent jihad," Bill Lewis, director of the FBI's Los Angeles field office, said Tuesday, Nov. 20, at a Los Angeles news conference. "The complaint alleges that they were willing to travel to kill what they considered the enemy."
The leader of the plot, federal officials said, was an Afghan native and former member of the U.S. Air Force who lived in Pomona. Through a mutual friend, he met two of the other men at a hookah lounge in August 2010 and later converted them to a violent, extremist version of Islam.
Sohiel Omar Kabir, 34, who was born in Afghanistan but became a U.S. citizen, was arrested Saturday in Kabul, Afghanistan. Kabir served in the Air Force in 2000 and 2001 and was honorably discharged, Air Force spokesman Lt. Col. Bret Ashworth said.
The FBI Joint Terrorism Task Force arrested the three others on Friday as they were leaving a home in Chino to drive to Mexico.
Apprehended were Arifeen David Gojali, 21, of Riverside; Miguel Alejandro Santana Vidriales, 21, of Upland; and former Cal State San Bernardino student Ralph Deleon, 23, of Ontario. Gojali was born in the United States. Deleon, born in the Philippines, and Santana, who was born in Mexico, are legal U.S. residents. Santana had applied for citizenship.
Each of the four men was charged with five felony counts, including conspiracy to kill, kidnap, maim or injure Americans using weapons of mass destruction. If convicted, the men could face a maximum sentence of 15 years in prison.
Additional charges could be filed, according to the U.S. attorney's office.
The four men said they would have attacked targets in the United States if they ever returned from Afghanistan, Dave Bowdich, special agent in charge of the counterterrorism unit in Los Angeles, said during the news conference.
"We weren't at that point," Bowdich added.
"We were on them early on," he said, referring to an informant who befriended the men. "We have kept close tabs on these individuals. Ultimately, we were able to pre-empt them from going over and harming our troops overseas."
They fantasized about becoming snipers, fighting on the front lines and blowing up U.S. military bases with semitrailer trucks loaded with plastic explosives, according to the federal complaint.
But the owner of a hookah bar hadn't seen that side of them.
Deleon, Santana and Kabir were customers of the Velvet Room, a hookah bar in Ontario, about two years ago, owner Adil Khan recalled Tuesday night.
"They were all good kids. They weren't extremists," Khan said. "I didn't notice anything out of the ordinary."
Deleon was a happy and cheerful man, Khan said.
Santana partied with Khan at the club, where customers sit on leather armchairs and couches while they smoke flavored tobacco from elaborate water pipes.
Kabir, the Air Force veteran, had suffered a head injury that left him with chronic headaches, Khan said. He described Kabir as a Raiders fan and something of a peacemaker at the club.
In the Upland cul-de-sac where Santana lived with his parents in an apartment, few neighbors knew the family, and none suspected he might have had ties to terrorism or al-Qaida.
Jen Collins, 24, who lived around the corner from Santana's apartment, said FBI agents showed up Friday with a sledge hammer and a crow bar. They searched the apartment all day.
She said she never thought anything was amiss in the weeks leading up to the raid.
"I was in utter shock. I don't know them. Nothing seemed out of the ordinary," Collins said. "I thought someone was getting raided. Never in a million years did I think there was a terrorist two doors down."
The 77-page federal complaint against the four lays out the men's plan to fight alongside Afghani jihadists working with al-Qaida against U.S. and coalition troops. The men discussed eight targets, including U.S. Air Force bases, United Nations operations, Jews and mercenaries.
The document also discussed how Santana and Deleon made trips to Southern California shooting ranges to practice firing assault rifles. The pair also visited a Corona paintball course in September to get the feel of paintball guns, which they thought resembled assault rifles, the complaint states.
Gojali, who went along on the paintball trip, was recruited a few days later to join the mission. The informant asked him if he was prepared to fight and kill.
Referring to the U.S. soldiers in Afghanistan, Gojali responded had watched video on the Internet on "what they're doing to our brothers and sisters."
"It makes me cry, I'm like so angered with them," Gojali said. "We are paying taxes for them so they can build arms and weapons to go train their soldiers."
The men began hiking in the mountains in order to get in shape for the rugged terrain of Afghanistan. They talked of hauling heavy loads on their backs as practice for carrying weapons such as rocket-propelled grenades
Deleon said he intended to go skydiving to understand how his body would react to fear.
Lewis said Kabir introduced the others to "radical and violent doctrine." He told them he had contacts with terrorist networks, including the Taliban and al-Qaida.
Kabir shared with Santana and Deleon the teachings of U.S.-born al-Qaida terrorist leader Anwar Al-Awlaki, who was killed by a U.S. drone missile in Yemen in 2011.
Santana and Deleon described Kabir as a trained terrorist who had come to Southern California to recruit other jihadists to joint the fight against America.
Kabir helped dissuade Santana from joining the U.S. military and persuaded him to fight for the Taliban instead. He said the U.S. military is "the true terrorist."
Santana said he was easily influenced growing up, and previously surrounded himself with gang activity. After listening to Kabir and al-Qaida teachings, he said he wanted to "fight for the right reason."
"Santana believed Islam was where he was going to be able to fit in and actually be able to fight for something that's right," according to the complaint. "Santana said that in his old life, he always looked for violence."
Muhamad Ali, an assistant professor of religious studies at UC Riverside and an expert on Islam, said converts are especially susceptible to extremist religious beliefs.
"They're very easily indoctrinated," Ali said. "Early conversion is fragile and prone to manipulation."
FBI agents first began monitoring the group in January 2012, after Santana was stopped entering the United States from Mexico while carrying a copy of the English-language, extremist-Islamic magazine "Inspire."
An undercover FBI agent contacted Santana online, where he discussed his plans to carry out jihad and join terrorist training camps in Afghanistan.
Santana and Deleon later met with the confidential informant over several months, and he related their activity to the FBI.
In March, Deleon said a couple of "brothers from the mosque wanted to join the jihad." He added that Santana and another man had been discussing fisabilillah, Arabic meaning "For the sake of God."
Santana said "Afghanistan had military bases that could be ambushed from every direction."
"I want to enjoy my targets. I wanna do C-4's if I could put one of these trucks right there. Just drive it into the baddest military base," Santana told the informant. "If I'm gonna do that, I'm gonna take out a whole base. Might as well make it like big, ya know."
In an online post, Santana pledged to become a sniper in Afghanistan. He said he had been training by shooting AK-47s and other weapons in Mexico. He also practiced making homemade explosives. He said his second choice would be using C-4 explosives.
Deleon said he wanted to specialize in explosives and fight on the front lines.
In order to prepare for their trip, Santana and Deleon video-conferenced over several months with Kabir through a Skype webcam. Kabir was living in Germany until he left for Afghanistan in late July.
Kabir told Santana and Deleon he was establishing ties with al-Qaeda and the Taliban in preparation for their arrival. He told the men that they would first have to work with the Taliban, which he referred to as "the students," and they would later work with al-Qaida, which he referred to as "the professors."
During their trip, they said they would wear Western clothing to look like tourists and carry sports magazines to avoid being detected as terrorists.
They would tell people they were going to Dubai to work in the cologne import/export business.
Kabir had made arrangements for them to stay in a home in Kabul and work at an orphanage as their cover.
Deleon withdrew from Cal State San Bernardino early and planned to cash out his remaining tuition and sell his car to pay for the trip.
Cal State officials said there was a student by that name who attended the university from the fall of 2007 until this year, when he voluntarily withdrew. He was a business administration major, studying marketing, and was close to graduating.
Santana and Deleon were ordered remanded to federal custody during a detention hearing Monday in Riverside federal court. They will return to court for a preliminary hearing and bail hearing on Wednesday, Dec. 5. Gojali's detention hearing was postponed until Monday.
The FBI document describes a scramble by Santana, Deleon and Gojali in the days leading up to their arrest to buy plane tickets from Mexico City to Istanbul and obtain a passport for Gojali.
After Gojali retrieved his passport form the Los Angeles federal building, he was shocked that it was issued.
"If they only knew," Gojali said.
Also contributing to this report: Staff Writers Richard Brooks, Richard DeAtley, Lora Hines, Imran Ghori Mark Muckenfuss and Brian Rokos.


text:
Our laminated budget folding custom can coolers fold flat for pocket or purse storage.
This logo can cooler by Koozie folds quickly for storage in pockets, purses, and bags.
The promotional can cooler insulator that started it all. One of the bestselling promotional products in the world.
Keep your drink cool at your next pool or beach party with this Collapsible Custom Can Insulator. With 8 different colors to choose from, your branding is sure to stand out in a crowd!
Thick polyurethane Foam Promotional Can Cooler will keep your beverages colder, longer.
Printed Koozie will maintain the cold temperate of your drink and the visibility of your company logo at your next outdoor event.
The Deluxe Collapsible Koozie logo can cooler is one of the bestselling promotional products in the world.
Collapsible promotional can cooler with heavy duty thick soft vinyl is just what you need during those sweltering summer days to keep your favorite beverage cold.
Collapsible Koozie promotional can cooler with no tear binding. One of the bestselling promotional products in the world.
The double wall construction of this Vacuum Promotional Tumbler & Can Insulator keeps your beverages hot or cold for hours! It's two product in one!
Sturdy polyurethane custom can cooler insulates your beverages and collapses to fit in your pocket.
A guaranteed summer smash, this promotional can cooler sleeve is valuable to your marketing efforts and inexpensive for your budget.
These promotional products from ePromos are ready to ship in 24 hours. Rush promotional items for a last minute event or if you're other supplier fell through.


text:
Only the finest ingredients make their way into Casper's Classics Sea Salt Caramel. Slow-Cooked natura sea salt carmael surrounded by the richest, creamiest Casper's Vanilla ice cream then hand-dipped in fine European Chocolate.
Picked fresh from the patch are the delectable strawberrys that are in Casper's Classics Dipped Strawberry. Plump juicy strawberries in creamy strawberry ice cream then hand-dipped in fine European Chocolate.
Take a cool creamy trip to the islands with Casper's Classics Creamy Coconut. Creamy delicisous vanilla ice cream with delectable shavings of island coconut all hand-dipped in rich dark Guittard Chocolate.
© 2012 Casper's Ice Cream - All Rights Reserved