Within the domain of brain studies, magnetic resonance imaging (MRI) provides unrivaled soft tissue contrast and is now the leading imaging modality for clinical research and care. It serves as a cornerstone for disease detection, precise diagnostics, and vigilant treatment monitoring across diverse age groups [1]. The distinctive feature of MRI lies in its remarkable capability to generate highly detailed 3-dimensional (3D) images, with a particular focus on capturing the intricacies of soft tissues, such as gray and white matters. This unique attribute positions MRI as an invaluable tool for delving into the complexities of the brain's internal structure and function [2]. Magnetic resonance (MR) images are acquired across diverse biophysical contrasts (e.g., T1, T2, and PD) and at different magnetic field strengths (i.e., 0.2 to 7T), each capturing specific characteristics of the underlying anatomy [3,4]. Consequently, higher field strengths, along with higher spatial resolution can reveal richer information and superior image quality of the brain tissue relative to images acquired at lower field strength and resolution.
Image-to-image (I2I) is a computer vision technique employed to enhance image quality and content. Within the field of MRI, it includes translation tasks such as one contrast to another within the same field strength (i.e., cross-modality) and from low-to high-field MR images for the same contrast. Although, this technique can be applied using both supervised and unsupervised approaches, supervised learning has shown higher performance as it enables the generation of high-quality images with sharp details and robust quantitative performance [5,6]. However, the requirement for paired datasets imposes a significant challenge as there is almost no accessible dataset available that includes paired MR images at both low and high field strengths for the same subjects and in multiple contrasts. For instance, the most widely used datasets in previous in the field of MRI include Alzheimer's Disease Neuroimaging Initiative (ADNI) 1 [7], Information eXtraction from Images (IXI) 2 , and datasets sourced from the Human Connectome Project (HCP) 3 , each of which has limitations. For example, in all mentioned datsets, only raw 3D MR images are presented, which necessitates intricate pre-processing steps including registration and brain extraction. Moreover, they include either MR images of paired subjects limited to a single contrast, or multiple contrasts but limited to one field strength.
To address this gap, we leverage the IXI dataset, which includes unpaired 3D MRI scans in T1, T2, and PD for different subjects at 1.5T and 3T. We propose a semi-synthesized dataset, PMC, which includes Paired Multi-Contrast MR images at 1.5T and 3T for the same subjects.
The PMC dataset is pre-processed and ready to use for supervised and semi-supervised learning methods in tasks, such as cross-modality, high-field MR image prediction, super-resolution, and multi-contrast MR image translation. This comprehensive dataset comprises MR images from 181 subjects, preciously crafted in both 2dimensional (2D) and 3D to accommodate a diverse range of models compatible with each of these formats. As Figure 1 represents, the dataset includes paired images in T1, T2, and PD contrasts at both 1.5T and 3T for each subject generated from the IXI dataset. In the 3D format, the total number of images in each contrast at each field strength is 181. All MR images across contrasts and field strengths for each subject are registered and have the same orientation. Additionally, the brain is extracted and the skull is removed.
In the 2D format, there are a total of 6576 images in each contrast at each field strength. These images are pre-processed and have the same size of 256×150. Similar to the 3D counterparts, they have undergone registration for a consistent orientation, brain extraction with skull removal, and augmentation using techniques such as flipping, rotation, scaling, and adding noise.
Furthermore, we provide a split version of the dataset for the 2D format. The entire dataset is divided into three subsets: the training set, the validation set, and the test set, with an as-close-as-possible ratio of 80% -10% -10%. Consequently, the data size for each contrast at each field strength is 5268, 648, and 660 for the training, validation, and test sets, respectively. To prevent models from exploiting subject-specific patterns in predictions, we ensure that no image from the same subject (including its augmentations) is distributed across different subsets. All versions of our proposed dataset will be released through our GitHub repository4 .
To create a dataset consist of MR images in multiple contrasts at both 1.5T and 3T for the same pseudo-subjects, a series of processing steps are undertaken as illustrated in Figure 2. Firstly, by leveraging demographic information from the IXI dataset, we meticulously select 181 subjects from the 1.5T set and 181 subjects from the 3T set, aiming for the closest possible match in terms of demographic details. Subsequently, subjects at 1.5T and 3T are paired based on matching sex, age, and ethnicity as closely as possible. The reason for pairing based on these demographic information is that aging, sex and ethnicity affect the brain overall structure and gray and white matter contributions [8].
Following this, MR images are reoriented to the standard orientation, cropped to dimensions of 256×150 to ensure uniform size, and reduce neck parts in the image with the aim of improving the brain extraction step. Subsequently, the brain is extracted and the skull is removed. We employ the FMRIB Software Library (FSL) 5 software for these tasks as it provides a comprehensive set of tools for image analysis and statistical analysis for functional, structural, and diffusion MRI brain imaging data [9].
Next, to generate MR images for the same subjects we follow two main steps: Firstly, T2-and PD-weighted MR images of each subject are registered to corresponding T1-weighted MR images at each field strength using rigid registration. It is worth mentioning that rigid registration is necessary for MR images of the same subject, due to the difference in the angle and position of the head during acquisition of the data. Secondly, 1.5T MR images are taken as reference and 3T MR images at each contrast are registered to their respective contrast at 1.5T using non-linear registration. For the registration steps, we utilize Advanced Normalization Tools (ANTs) 6 software as it is widely recognized as an advanced medical image registration and segmentation toolkit that effectively manages, interprets, and visualizes multidimensional data [10]. Also, it should be noted that all aforementioned processing steps are applied to the 3D MR images, resulting in PMC dataset in 3D format.
Moreover, to extend the data generalizability to networks solely employing 2D data and increase the number of samples, 3D MR images are transformed to 2D. Specifically, we select slices that predominantly contain the brain (i.e., 10 slices per 3D MR image) while avoiding slices with minimal or no brain content. Additionally, to increase the size and generalizability of the dataset, data augmentation techniques, including flipping, rotation (with an angle of ±5 degrees), noise addition (e.g., Gaussian with random standard deviation in range of [5,10] and salt-and-pepper with a probability uniformly sampled from the interval of [0.05,0.1]), and scaling (with a factor of 1.2) are applied. As a result, the data size for each contrast at each field strength increased to 6576.
To assess the quality of the synthesized MR images at 3T compared to the reference images at 1.5T, we first employ evaluation metrics including mean squared error (MSE), peak signal-to-noise ratio (PSNR), Pearson correlation (CORR), and mutual information (MI) [11]. We compare the synthesized 3T images with corresponding reference images at 1.5T as there are no labels available at 3T for checking the synthesis quality. Thus, utilizing these metrics, we assess how close 3T images are synthesized compared to 1.5T ones in terms of contrast and overall structure as reported in Table 1.
Synthesized MR images at 3T compared with the reference images at 1.5T evaluated using MSE, PSNR, CORR, and MI metrics (The directions of vertical arrows indicate higher image quality. Results are reported as the mean±standard deviation). However, it should be noted that in MR images acquired at 1.5T and 3T even for the same contrast, there are differences in the relative signal intensities in gray and white matter and accordingly in the resulting output contrast [12]. Consequently, to investigate the quality of the synthesized images and minimize the impact of contrast differences during evaluation, we conduct morphologybased comparative analyses which have been proven to be reliable in the state-of-the-art studies in related fields [13]. We extract the morphological patterns of images (using edge detection techniques) at both 1.5T and 3T for each contrast as shown in Figure 3 to assess whether the patterns and morphology of the synthesized data at 3T align with the reference data at 1.5T. Next, we evaluate the extracted patterns using MSE and structural index similarity measure (SSIM) [14] as reported in Table 2. Also, to compare the synthesized images with references within different spatial frequency ranges and accordingly different levels of details, we perform 2D wavelet analysis on the synthesized images and corresponding references to decompose them into four different frequency components and select the three most high frequency ones named as Subband 1, 2, and 3, respectively [15] as Figure 4 illustrates. Table 3 displays the subband-wise comparative results.
Extracted Pattern 1.5T 3T
The PMC dataset can be applied in a wide range of tasks involving MR image translation, in particular, image generation, different stages of model development, and pretraining models for small target dataset sizes. In the following, we investigate the capability of our dataset in supervised methods for the aforementioned tasks.
U-Net is one of the most commonly used neural networks for tasks such as cross-modality, super-resolution, and multi-contrast MR image translation [16,13,17,18]. Thus, to further investigate the application of the proposed dataset, a U-Net based architecture, which was previously proposed in [17] and has shown high performance in the mentioned applications, is implemented in this paper for the following tasks:
1. Cross-modality MR image translation 2. 3T MR image prediction from the same contrast at 1.5T 3. 3T MR image prediction using 1.5T multi-contrast MR images Table 4 displays the results for image generation in each task using the PMC dataset, indicating the highest performance in Task 1 and 1.5T T1 to 1.5T T2 translation.
Quantitative results of generated MR images using U-Net compared with the ground truth images, using PMC dataset (The directions of vertical arrows indicate higher image qualities. Results are reported as the mean±standard deviation). Moreover, to investigate the effectiveness of the PMC dataset in developing models based on cross-dataset evaluation scenarios, we utilize the latest release of the Open Access Series of Imaging Studies (OASIS) 7 , known as OA-SIS3 dataset [19], which includes MR images at 1.5T and 3T in T2, for Task 2 (3T MR image prediction from the same contrast at 1.5T ). First, we train and test the model on the OASIS3 dataset. Then, to compare the effectiveness of using the PMC dataset, we use it to train the model and test the model on the OASIS3 dataset. The results for both approaches shown in Table 5, suggest that our dataset demonstrates acceptable performance. Specifically, the U-Net, demonstrates higher efficacy when trained on PMC for 1.5T T2 to 3T T2 MR image translation.
In this study, we introduced the PMC dataset, which consists of paired MR images in multiple contrasts of T1, T2, and PD and at both 1.5T and 3T field strengths for the same subjects. The dataset is pre-processed and presented in 3D, 2D, and a split version of 2D, ensuring compatibility with a wide range of models and application in image translation tasks within MRI. Quality evaluation of the proposed dataset involved the use of MSE, PSNR, CORR, SSIM, and MI evaluation metrics, along with morphology-based methods. We also demonstrated the applicability of the data for supervised methods, particularly in cross-modality MR image translation, 3T MR image prediction from the same contrast at 1.5T, and 3T MR image prediction using 1.5T multi-contrast MR images. Moreover, we highlighted its extendability to cross-dataset evaluation scenarios.