Human brain functional MRI and DTI visualization with virtual reality
1Department of Engineering, Purdue University Calumet, Hammond, IN 46323; 2CIVS, Purdue University Calumet, Hammond, IN 46323
|
Cite this article as: Chen B, Moreland J, Zhang J. Human brain functional
MRI and DTI visualization with virtual reality. Quant Imaging Med Surg
2011;1:11-16. DOI: 10.3978/j.issn.2223-4292.2011.11.01
Original Article
|
|
Abstract
Magnetic resonance diffusion tensor imaging (DTI) and functional MRI (fMRI) are two active research areas in neuroimaging. DTI is sensitive to the anisotropic diffusion of water exerted by its macromolecular environment and has been shown useful in characterizing structures of ordered tissues such as the brain white matter, myocardium, and cartilage. The diffusion tensor provides two new types of information of water diffusion: the magnitude and the spatial orientation of water diffusivity inside the tissue. This information has been used for white matter fiber tracking to review physical neuronal pathways inside the brain. Functional MRI measures brain activations using the hemodynamic response. The statistically derived activation map corresponds to human brain functional activities caused by neuronal activities. The combination of these two methods provides a new way to understand human brain from the anatomical neuronal fiber connectivity to functional activities between different brain regions. In this study, virtual reality (VR) based MR DTI and fMRI visualization with high resolution anatomical image segmentation and registration, ROI definition and neuronal white matter fiber tractography visualization and fMRI activation map integration is proposed. Rationale and methods for producing and distributing stereoscopic videos are also discussed.
Key words
Human brain visualization; stereoscopic 3D visualizationQuant Imaging Med Surg 2011;1:11-16. DOI: 10.3978/j.issn.2223-4292.2011.11.01
|
|
Introduction
Human Brain DWI and DTI Visualization
Methods for visualizing the human brain have evolved in recent years. Traditional methods of visualizing volume brain data include functionality for segmenting volumetric head data into brain, skull, etc., viewing 2-D slices and rendering and manipulating surface models (1). The other dominant visualization method has been volume rendering, including volume texture mapping, splatting, and other techniques (2). More recent work has explored the use of streamlines and streamtubes for visualizing brain fibers (3,4).
The principles of diffusion tensor imaging (DTI) (5-8) involve the acquisition of diffusion-weighted images sensitized in various gradient directions, with one or more encoding levels in each direction, followed by pixel-by-pixel calculation and diagonalization of the diffusion tensor (9,10). The diffusion tensor provides two new types of information of water diffusion: the magnitude and the spatial orientation of water diffusivity inside the tissue. Whereas brain gray matter has weak anisotropy in tensor orientations, myelinated axons have shown strong tensor orientation (11-13). The anisotropic information, therefore, is exploited by a new technique called DTI fiber tracking or tractography to determine neuronal fiber bundles in a noninvasive manner which opens a new way to assess white matter development and pathology (14).
Functional BOLD Signal and Brain Activation Maps
During brain activation, metabolic activities in certain regions increase and consequently require more oxygen and glucose (15,16). The cerebral blood volume (CBV) and cerebral blood flow (CBF) increase correspondingly in order to remedy the deficit of oxygen and glucose. Because the deoxyhemoglobin has paramagnetic susceptibility, it induces magnetic field changes in the intravascular and extravascular spaces and increases T2*. The susceptibility-related intravoxeldephasing decreases and the spin
coherence increases, resulting in increased MRI signal intensity
(17-19). The functional contrast on the blood oxygen leveldependent
(BOLD) mechanism can be statistically calculated
with a linear model by multiple linear regression algorithms. The
regions with higher significance scores will be treated as activated
regions. The calculation of brain activation can be parallelized to
dramatically reduce the computing time (20,21).
Human Brain Diffusion Tensor and Activation Map Visualization
Methods for visualizing the human brain have evolved in recent
years (22-26). Traditional methods of visualizing volume brain
data include functionality for segmenting volumetric head data
into brain, skull, etc., viewing 2-D slices and rendering and
manipulating surface models. The other dominant visualization
method have been volume rendering, including volume texture
mapping, splatting, and other techniques. More recent work has
explored the use of tensor visualization instead of traditional
streamlines and streamtubes for brain fiber visualization (27-31).
In addition to visualization techniques, display technology
and virtual reality has also evolved on multiple fronts including
stereoscopic displays, head mounted displays, and immersive
projection systems like the CAVE and its derivatives (31).
Some attempts have been made to use these types of displays in
presurgical planning (32,33). However, many of these efforts are
highly specialized and there are still gaps in research involving
brain visualization on virtual reality systems for basic research,
education and presurgical planning. In this study, virtual
reality (VR) based MR DTI and fMRI visualization with high
resolution anatomical image segmentation and registration,
ROI definition and neuronal white matter fiber tractography
visualization and fMRI activation map integration is proposed as
well as a new function for remote stereoscopic 3D visualization.
The main purpose of the application is for basic research, clinical
applications and education. This work builds on previously
proposed developments in DTI with VR (34).
|
|
Methods
DTI Data Acquisition
The diffusion tensor imaging data were acquired using a singleshot
diffusion-weighted SE EPI sequence with repetition time
TR=2000 ms, time of echo TE=61.6 ms, field-of-view (FOV)=25.6 cm, matrix size=128×128, slice thickness=2 mm, an
axial slice orientation and left/right as the frequency encoding direction. To reduce the acquisition time and signal to noise
ratio, a SENSE reduction factor of 2, ramp sampling, and 5/8
partial Fourier encoding were used to minimize the readout
window and TE, and hence the geometric distortions and signal
loss due to T2 relaxation. Diffusion-weighting was applied along
15 non-coplanar directions evenly distributed over a sphere with
a b-factor of 800 s/mm2, and 10 averages were used to increase
the SNR.
The B0 field map data were collected for geometric distortion
correction using identical TR, TE, FOV, slice thickness, and
diffusion-weighting scheme with lower resolution and multiple
echoes. All B0 maps were acquired on a 20 cm diameter spherical
uniform phantom, whereas only a non-diffusion-weighted B0
map was acquired in vivo.
The diffusion tensor was then computed back from the
corrected diffusion weighted images with theLevenberg-
Marquardt nonlinear fitting algorithm. The tensors were
decomposed into eigenvectors and eigenvalues for DTI fiber
tracking.
The DTI and fMRI experiments were conducted on a 3T
Tesla General Electric SignaHDx MRI scanner (GE Healthcare,
Milwaukee, WI) at Purdue University Research Park. The
Institutional Review Board (IRB) was obtained through Purdue
University for the research involving human subjects. The
datasets were crosschecked with the datasets acquired from other
MRI scanners (Philips or Siemens scanners) in DTI, stimulus
fMRI experiments with identical image acquisition parameters.
Functional MRI data acquisition and activation map computation
Total acquisition time was approximately 2 hours including the
preparation time with 6 runs in a session.
(i) Acquire a high-resolution whole brain image using a T1 weighted MPRAGE sequence with 1 × 1 × 1 mm voxel size. (ii) Acquire images using a single shot EPI sequence with TR=2 s, TE=30 ms, matrix size 64 × 64, voxel size: 3mm × 3mm × 5mm, and slice thickness=5 mm (no gap). The number of slice is approximately 28 for the whole brain coverage. (iii) Monitor the physiological parameters, respiratory and cardiac signals as well as head. (iv) movement during functional data collections. Block design of rotating checkboxes with 6 runs for fMRI experiments with stimulus. Block design of rotating checkboxes with 6 runs for fMRI experiments with stimulus. The functional MRI data were processed with the following
steps:
(i) Image re-alignment and head motion correction (ii) Spatial normalization and smoothing (iii) Statistical model specification (iv) Activation map generation and mapping Activation maps were calculated with Statistical Parametric Mapping (SPM). Virtual Reality and Human Brain Visualization
Several different software packages were used in the initial
development of the various brain models and the VR application.
Initially, surface models were exported as VRML and converted
to OpenSceneGraph for integration with the VR application
(35,36). The surface models were then registered to fit within
a volume rendered brain and skull. The volume rendering was
produced with osgVolume, using DICOM data.The virtual
reality application was developed using OpenSceneGraph for
both surface models of the segmented brain data and volume
rendering of composite brain. The application was originally
developed for a two-screen immersive projection system as seen
in Figure 1. The system included 7'6"x10' screens, using passive
projection and infitec glasses for stereoscopic 3-D effect. The
system also utilized an Intersense IS-900 6-DOF tracking system
with head-tracker and wand (37).
In this iteration of the software, using the wanda user could
select individual segments of the brain and display both the appropriate surface model (enclosed within the partially opaque
volume rendering of the composite brain) along with the correct
brain segment name. Selection was done using raycasting along
with some OpenSceneGraph functions (38).
Eventually, Avizosoftwarewas used to develop additional
functions in this project so that Purdue Calumet and Purdue
West Lafayette share the same framework for visualization (39).
Instead of rendering pre-calculated fibers in low accuracy, the
request was passed to a Linux cluster for on-demand neuronal
tracking and functional connectivity for high accuracy and
flexibility.The quality of calculated neuronal fiber tracks were
evaluated using the Fréchet distance against the standard MRI
templates. Well studied neuronal fiber bundles such as uncinate
fasciculus, superior longitudinal fasciculus and corpus callosum
were used for validation.
Figure 1. The configuration of the VR system for the initial
developments of the software.
|
|
Results
Figure 2A shows the transparent and volume rendered human
brain with the color coded corpus callosum with the “see
through” effect as well as the intersected 2D sagittal slice. Figure 2B shows the sliding slice inside the brain and can be served as a
reference image. These visualization technique can be displayed
at the same time.
Figure 3A is surface rendered deskull brain with color coded
segmented regions. Figure 3B and 3C show views of the brain
after “taking off ” the superior frontal gyrus (SFG), superior
temporal gyrun (STG), middle temporal gyrun (MTG) and
postcentral gyrus in the parietal lobe.
The remote stereoscopic 3D visualization is demonstrated
with video samples on YouTube 3D. Video is initially captured
from Avizo in a Side-by-Side format (using Avizo's built-in
stereoscopic viewing functionality). The aspect ratio of the video
needs to be corrected to accommodate Youtube's 3D format
requirements.
Figure 2. (A) Human brain volume rendering and color coded region visualization. The 2D sagittal slice provides additional reference views. (B) The slice can slide inside of the brain and can be turned on or off by users.
Figure 3. A: The surface rendered deskull brain with color coded segmented regions; B/C: the views of the brain after “taking off” the superior frontal gyrus (SFG), superior temporal gyrun (STG), middle temporal gyrun (MTG) and postcentral gyrus in the parietal lobe.
|
|
Discussion
The benefit of stereoscopic viewing for medical applications
has been discussed in recent publications (40,41). However a
widespread adoption of stereoscopic viewing of 3-D data across
medical domains is still lacking. We have adopted a workflow
that provides the option of stereoscopic viewing to the user
regardless of their display capabilities. It is then at the user's
discretion whether to view monoscopically or stereoscopically.
While most 3D visualization packages, such as Avizo,
provide functionality that allows the display to be output in
stereoscopic 3D, this functionality still depends on the end-user possessing a stereoscopic display. Additionally, if one wishes to
share visualizations with someone who doesn't have the Avizo
software, there are limited options for stereoscopic content
sharing. Generating anaglyph for colored glasses provides a
quick way to distribute 3D contents on traditional displays. With
the rise of 3DTV and increased supply of stereoscopic content
in popular culture (42), an additional method of stereoscopic
deployment now available through Youtube's 3D content
capabilities (43). Due to the in-progress nature of Youtube's 3D
functionality however, the methods used to create and distribute
videos from Avizo has changed and may change again as the
stereoscopic community provides additional feedback to the
Youtube 3D developer.
In conclusion, an immersive visualization application
for MR DTI, neuronal white matter fiber tractography and
fMRI activation map visualization have been proposed
and implemented. Visualization techniques include surface
modeling, volume rendering, and streamlines or streamtubes.
The application will have potential applications in basic research,
education, and surgical planning. Continued development is
needed to implement the tensor visualization, 3D stereoscopy
and interactive visualization.
|
|
Acknowledgment
The authors would like to thank the support from the Center for Innovation through Visualization and Simulation (CIVS) at Purdue University Calumet during the course of this work.
|
|
References
|
