Prostate segmentation in 3D MRI

Outlines the prostate in a new 3D MRI scan automatically by reusing outlines an expert drew on 15 earlier patients: atlas-based segmentation with rigid, affine and B-spline registration in Elastix. Team project; I built the registration pipeline, which reached a mean Dice of 0.83 on unseen patients against 0.75 for the team's U-Net.

When
Early 2023
Where
MSc coursework, Capita Selecta in Medical Image Analysis at TU Eindhoven, in a team of five
Built with
Python, Elastix, SimpleITK, NumPy, SciPy, Image registration, PyTorch (team)
Made as
Team project

About the project

The problem. Before radiotherapy for prostate cancer, a doctor outlines the prostate on every slice of a 3D MRI scan, so the radiation can be aimed at it while sparing the bladder and rectum next to it. Marking which part of a scan belongs to an organ is called segmentation. By hand it takes a long time, so our team built two ways to do it automatically and compared them.

The data. We had MRI scans of 15 earlier patients, each a stack of 86 slices. On every slice where the prostate appears (41 to 78 slices per patient), an expert had already traced its outline. A scan together with its expert outline is called an atlas.

Atlas-based segmentation (my part). The idea is to reuse the expert's outlines instead of teaching the computer what a prostate looks like. An atlas outline cannot simply be copied onto a new scan, because every body differs in position, size and shape. So the atlas scan is first reshaped until it lines up with the new scan. This step is called image registration. The outline is then reshaped in exactly the same way and lands on the new patient's prostate; this is called label propagation.

Registration runs on the full 3D volume with the Elastix toolkit, in three stages. Each stage starts from where the previous one ended and is allowed more freedom:

  • Rigid: shift and rotate the whole scan (6 parameters).
  • Affine: also scale and shear it, to match overall size and proportions (12 parameters).
  • B-spline: lay a grid of control points over the volume and move each one separately, so the scan can bend locally, for example where a full bladder presses on the prostate (4,704 parameters here).

To know when the images match, the computer uses mutual information, a score that rises as the intensities of the two scans line up. An optimiser (adaptive stochastic gradient descent) nudges the parameters a little, checks the score, and keeps whatever improves it. It repeats this hundreds of times, first on a blurred, low-resolution version of the scans and then at full detail. Nothing is trained in advance: the reshaping is solved fresh for every new patient.

From 15 outlines to one. This is done separately for each of the 15 atlases, which gives 15 propagated outlines on the new patient. They mostly agree but are never identical, so they are combined. This step is called label fusion. We tried two methods: majority voting, and weighted fusion (Isgum et al., 2009), in which an atlas gets a bigger vote in the areas where its reshaped scan matches the new scan best. These weights are computed again for each new patient, not learned.

Registering all 15 atlases is slow, so I also tried atlas selection: rank the atlases by how similar they look to the new scan (SSIM or normalised mutual information) and register only the closest few.

Measuring the result. Quality is measured with the Dice score, the overlap between the computer's outline and the expert's: 1 means identical, 0 means no overlap. To test fairly, we used leave-one-out: each of the 15 patients took a turn as the "new" patient, with the other 14 as atlases.

  • All atlases, weighted fusion: median Dice 0.83, the best setup.
  • Only the three most similar atlases (by SSIM), majority voting: median Dice 0.81, in 5.3 minutes instead of 16.5 for five patients.

The deep learning comparison (teammates). Three teammates trained a U-Net, a neural network that learns to segment from examples, on 13 of the patients. A variational autoencoder (VAE) generated extra synthetic slices to make up for the small dataset. This raised the U-Net's validation Dice from 0.52 to 0.55.

The result. On five patients that neither method had seen, atlas-based registration scored higher on every patient: mean Dice 0.83 against 0.75. With only 15 examples, reusing an expert's outlines works better than learning from scratch. The trade-off is speed: once trained, a network gives its answer in a single pass, while registration has to solve 15 optimisations for every new patient.

My part. I built the registration side: the rigid, affine and B-spline stages in Elastix with full 3D output, and a penalty setting that removed stripe artefacts around the propagated outlines. I also wrote the atlas selection by SSIM and NMI, and the Dice and precision evaluation. A teammate implemented the two label fusion methods. I joined both parts into one pipeline and produced the results, including the test on the five unseen patients.

More from Computer vision, AI and image processing

Get in touch

Let’s talk if you have a project in mind.