The field of Technology Guided Therapy (TGT) combines a number of different medical diciplines, including Medical Imaging, Image Registration, Image Segmentation, and Surgical Data Collection and Processing.

Preoperative Medical Imaging

Preoperative imaging provides three-dimensional patient-specific information about anatomy, function and the location of both diseases and healthy structures. Preoperative imaging consists of tomographic sets such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET) and Single Photon Emission Tomography (SPECT). Each of these modalities provides different information about the structure and/or components of the patient's body.

Specific applications of MR tomograms include MRA (Magnetic Resonance Angiograms) which are MR images made in such a fashion as to be sensitive to flowing blood and functional MR (fMR) which are MR images which are sensitive to the changes in the brain associated with increased use. In CT, a relatively new technique called spiral scan is allowing for much faster imaging. This speed of imaging means that contrast agent can be injected into the patient's blood stream and the vascular bed of interest can be imaged before the contrast washes out. This is known as CT Angiography or CTA.

Image Registration

Registration is the determination of a mapping between the coordinates in one space and those in another, such that points in the two spaces that correspond to the same anatomic point are mapped to each other. Mappings may be classified as rigid or non-rigid, where rigid mappings are those that preserve all distances. Because of the rigidity of bone and the relative rigidity of anatomy that is attached to bone, as in particular the contents of the skull, rigid mappings are of special importance. Rigid mappings may be specified in terms of a translation and a rotation.

Three-dimensional cross-sectional images provided by such modalities as CT, the many modes of MR, and PET have the potential to provide the physician with complementary sets of information about anatomy and function. Multiple images of the same modality have the potential to provide information about disease progression, and before-and-after images can demonstrate the efficacy of treatment. When more than one image is ordered, however, the problem arises as to how the information from the physically separate images can be combined. While much of the information is properly combined in the mind of the physician, there is often uncertainty as to precisely which points correspond, especially at the scale of a few millimeters. When one or more images are consulted during surgery the uncertainty extends to the correspondence between the images and the anatomy itself. The problem of determining the corresponding points between two images or between an image and the anatomy that gave rise to the image is the problem of image "registration". Registration problems can be partitioned into image-to-image registration and image-to-physical-space registration, the former problem arising primarily in diagnosis, the latter in image-guided therapy.

The problem of registering three dimensional tomographic images has been a subject of research since at least the early eighties [1], and recently, if publication statistics are any indication, there has been an explosion of interest. By 1993 over 200 papers had been published on medical image registration [2]. A survey to be published early this year will list about 300 additional papers published since 1993 [3]!

Intrasurgical Guidance, Image Display and Data Gathering

Where Technology-Guided Therapy moves from being an imaging or image processing field to a therapeutic process is in the guidance process. Here the images are used, not as pictures but as maps. Inherent in this process in that the surgeon not only sees where his or her instruments are but where they are relative to both the lesion (or site of surgical interest) and to sensitive, healthy structures that the surgeon would like to avoid.

For this process, three dimensional spatial localizers (3DSLs) must be used. While any number of devices have been tried over the years, at the Center for Technology-Guided Therapy we have had two classes: Articulated Arms and Optical Tracking Systems.

Articulated Arms

The 3DSL we first developed was an articulated arm with six-degrees of freedom. The six degrees of freedom were rotational joints whose angular positions were sensed by optical encoders . The six degrees of freedom allowed any point in the work envelope to be approached at any trajectory and the optical encoders allowed unprecedented angular resolution (0.005 degrees). Given that high degree resolution, we had to invent a novel calibration technique which has since found applications in the general robotics community [Edwards and Galloway]. Engineering rigor on issues such as lateral runout in the bearings and backlash in the gears, gave this device, dubbed the Mark I, submillimetric mechanical accuracy over its entire work envelope. The measurements were made on a series of precision machined phantoms. The mechanical accuracy was tested over virtually every possible path to the test points. A point would be touched and the next point localized by completely twisting the arm into a new configuration. We were rigorous in requiring that every degree of freedom be different between measurements. Thus we were never working in just a "local space". This is crucial to allow the surgeon to have confidence in the performance of the device regardless of the path to the point dictated by surgical constraints. We distinguish the performance of the localizer, called mechanical accuracy, from the intrasurgical guidance performance, called application accuracy. The application accuracy is influenced by scan thickness, registration and patient motion in the scanner. We performed over 50 IRB approved procedures with the Mark I and our average application accuracy was 1.58mm or less than a voxel in size.

The Mark 1 (Left) and Mark II (Right) Articulated Arms

The Mark I proved to be a surgical guidance success but its prototype nature made for poor ergonomics. We designed the Mark II (shown above) of 6061-T6 aluminum and provided a counter-weighting system to allow one-hand operation. Although the Mark II was longer (80 cm) as compared to the Mark I's 60 cm, its mechanical accuracy of 0.7 mm was better than the Mark I's. With the Mark II we provided submillimetric mechanical accuracy over a larger work envelope with "weightless" operation. But the device still had mass. When the surgeon started it moving or stopped it, he or she had to overcome the device's inertia. The surgical user's opinions were mixed as to whether or not the inertia was a problem. One of the IIGS pioneers and one commercial developer used articulated arms with potentiometer-based angular measurement to reduce the arm weight at the cost of mechanical accuracy. Three other groups have developed articulated arms all of which weigh more than the Mark II. It finally became our decision that we had to change paradigms for 3DSLs.

Optical Tracking

To address the issue of inertia we have changed localization paradigms and are now using an optical triangulation system. In a 1987 paper, Horn, demonstrated a closed-form solution based on unit quaternions for mapping an object from one reference frame to another. This technique requires that at least three non-collinear reference points on that object be localized in each frame of reference. Horn shows the technique returns a transformation solution which minimizes the sum of square error and the accuracy in determining the transformation is improved by using more than three points. The use of quaternions has the additional advantage of being computationally simple thus allowing rapid calculation. Given a probe with multiple, non-collinear reference points, that probe's position and orientation can be located and tracked in any coordinate system. If a large enough number of reference points are placed on the probe to guarantee that at least three may be localized given any probe orientation, then the surgeon is freed from the constraint of having to hold the probe in a manner defined by the system.

There are three costs inherent in the use of a probe with a large number of reference points. The first is one of probe localization speed. Since probe localization requires N reference localizations, as N grows probe localization slows. This constraint argues for techniques in which the emitters can be quickly localized. The second cost is one of weight, adding reference objects to a probe will increase its weight. Finally, unless all reference points can be perceived at all orientations, the algorithm used to determine the transformation matrix from one space to another must be able to handle missing values. We have constructed a set of cylindrical probes, one style with 24 and another with 25 infrared light emitting diodes (IREDs) spiraling around the handles. Please see the video. This configuration allows a subset of the IREDs to be perceived by the sensor unit, irrespective of probe's orientation. One type of probe (25 emitter) has several IREDs oriented directly away from the tip of the probe. This "rear-firing" eliminates a dead zone perpendicular to the long axis of the probe. One of the probes is shown in surgical use below.

The 25 IRED probe in use in surgery. A screen shot shows the position on a CT (top left), an MR-T1(top right). an MR-T2 (bottom left) and a PET scan (bottom right)

By using the IREDs, which weigh less than 3 grams each with wire and mounting hardware, as reference points, we could construct probes which weighed less than 100 grams. In order to determine the location of each of the IREDs we are using an Optotrak/3020 (Northern Digital Inc, Waterloo, Ontario) position sensor. The position sensor (shown below) contains 3 linear CCD devices containing 2048 elements, with cylindrical optics placed in front of the CCDs to compress the field of view into a single line. As the infrared light from each IRED falls on the linear CCD it excites a number of the CCD elements in the array. This signal is captured and the background illumination is removed by the application of a threshold. The processed linear signal now represents the spread function of the IRED in the detector's plane of sensitivity. The centroid of the linear signal is determined and the position of that centroid is used to locate the object in the CCD's plane of view. By using the position of the centroid, instead of the position of the maximum CCD element, the IRED's location can be resolved to a much finer sampling than the CCD element spacing. If the signal from each CCD element falls below the threshold or if the total linear signal fails to meet a quality criteria due to IRED angulation or intensity issues, the IRED is reported as missing. Each IRED is fired in time sequence and the sensor unit returns either a "missing" value or the position of each IRED in 300 microseconds. This allows a 25 IRED probe to be easily tracked at 40 Hz.

The Optotrak 3220 localizing planes

Once the probes have been constructed, they are calibrated by locating each of the IREDs and the endpoint in a reference coordinate system. A reference file is constructed containing these locations. In order to perform a probe localization, the "non-missing" IRED locations reported by the sensor unit are matched to the corresponding reference locations and a transformational matrix is formed according to the methods of Horn. The reference location of the probe endpoint is then passed through the transformational matrix and the endpoint's position in sensor space is calculated. A "goodness of fit" parameter is calculated from the residuals allowing the rejection of poor transformations due to barely acceptable measurements.

While the optical probe and sensor unit provides for most of our concerns (probe weight, probe localization speed, and orientation sensitivity), it still requires that the sensor unit be rigidly attached to the patient to maintain a constant coordinate system. Since the sensor unit weighs over 40 kg, this would considerably reduce positioning flexibility in the OR. The problem was addressed by creating a second perceptible object called the Reference Emitter, this one stationary relative to the patient. The Reference Emitter is a small rectangle with 6 IREDs (Please see operating room picture above) which can be located in sensor space in a manner similar to the probes.

Once the Reference Emitter has been localized, the probe's position can be mapped into a co-ordinate system defined by the Reference Emitter and the position of the sensor vanishes from the equations. This frees the sensor unit from having to maintain a fixed location to the OR table. Thus we can move the largest element in the system, the sensor unit, around the operating room during the case. This greatly reduces the interference of the device with the surgical process. Only the very light Reference Emitter must be mechanically coupled to the head restraint device.

Image Display

At Vanderbilt we pioneered what has become the standard format for Image-Guided Displays, the 4 quadrant display. In the first image below, we show an image from 1989 with 4 512x512 images. In this case the four images are a native transverse CT image, and two reconstructed images in the cardinal planes: coronal and sagittal. On the lower right, there is a wireframe which was our first attempt at providing three-dimensional trajectory information.