health

TissueNet: Detect Lesions in Cervical Biopsies

The objective of this challenge is to detect the most severe epithelial lesions of the uterine cervix. The challenge brings together thousands of microscopic slides from different medical centers across France. #health

€25,000 in prizes
oct 2020
540 joined

The objective of this challenge is to detect the most severe epithelial lesions of the uterine cervix. The challenge brings together thousands of microscopic slides from different medical centers across France.

Quick Facts

Participants
540
No. of Entries
607
Prize
€25,000
Winner

karelds

1st Place

Navigation

Data resources

Whole slide images are digital formats that allow glass slides to be viewed, managed, shared, and analyzed on a computer monitor. These extremely high resolution images require special software to be able to read and manipulate in memory. A few different packages exist to do this in Python, and we will cover the main ones here. Each WSI image in the train set is also available as a pyramidal TIF, so we'll cover some helpful tips for that format too. All test images are pyramidal TIFs.

Formats

Native whole slide images

The native whole slide image (WSI) formats present in our dataset are:

  • .mrxs (MIRAX)
  • .svs (Aperio)
  • .ndpi (Hamamatsu)

These formats are related to the scanner used to create the images. These are provided in the native resolution and magnification. You can determine the exact resolution and magnification by referring to train_metadata.csv or test_metadata.csv. While the pyramidal TIF format has standardized, precomputed zoom levels, the available levels for WSI can vary. You can determine the zoom levels for each image using OpenSlide.

Pyramidal TIF

We recommend working from the pyramidal TIF versions of the slides in your machine learning workflow. The software tools for this format are better maintained and less prone to error than those available for WSIs.

Pyramidal TIFs are a multi-resolution, tiled format. Each resolution is stored as a separate layer or "page" within the TIF. Page 0 contains the image at the full resolution of the WSI. Subsequent pages contain lower resolution images―each layer is the previous layer downsampled by a factor of two, until the largest dimension of the downsampled image is less than 512 pixels. In other words, relative to the full resolution, the pages are downsampled 2x, 4x, 8x, 16x, and so on. Each layer is stored as individual 512x512 "tiles" where each tile is compressed using JPEG with quality (Q) 75.

TIF illustration

An illustration of the multi-scale property of the pyramidal TIF image format. Source: Cmglee - Own work, CC BY-SA 3.0, Link

Using the full-resolution provided in the metadata csv, you can calculate the resolution of each layer as full_image_resolution * (2 ** page). The resolution reported in the metadata CSVs is in the units microns/pixel, so larger numbers are more zoomed out.

Given the enormous size of the full-resolution images, which makes it impossible to load even a single image into memory, you will need to get creative in your use of multiple scales! For example, one of the first things you will notice browsing the images is that much of the slide is blank space with no tissue whatsoever. It would be a waste of your limited compute time to analyze those parts at full resolution. One simple solution is to segment the image into tissue/non-tissue at a lower resolution and analyze only the tissue regions at higher resolutions.

Libraries

OpenSlide

OpenSlide supports all native whole slide image formats, including:

  • .mrxs (MIRAX)
  • .svs (Aperio)
  • .ndpi (Hamamatsu)

Here's a code snippet to perform some basic image operations using OpenSlide:

import openslide

# load the slide
slide = openslide.OpenSlide("slide1.ndpi")

# number of levels
print(slide.level_count)
>> 8

# dimensions for each level in the image
print(slide.level_dimensions)
>> ((67456, 84224), (33728, 42112), (16864, 21056), (8432, 10528), (4216, 5264), (2108, 2632), (1054, 1316), (527, 658))

# magnification for each level
print(slide.level_downsamples)

# highest resolution available in micrometers per pixel
print(slide.properties["openslide.mpp-x"], slide.properties["openslide.mpp-y"])
>> 0.22717462913741793 0.22717462913741793

# generate thumbnail
thumbnail = slide.get_thumbnail((output_height, output_width))

# read an image region in as a PIL Image
x, y = 100, 200
level = 2
region_width = 500
region_height = 500
region = slide.read_region((x, y), level, (region_width, region_height))

# convert the PIL Image to a numpy array
array = np.array(region)

# red, green, blue, alpha channels
print(array.shape)
>> (500, 500, 4)

OpenSlide Python is unable to read in the pyramidal TIF formats. To do that, you should use PyVips.

PyVips

PyVips is a Python binding for libvips, a low-level library for working with large images. PyVips can be used to read and manipulate the pyramidal TIF formats. Here's a code snippet to perform some basic image operations using PyVips:

import pyvips

# load the full-resolution image
path = "slide1.tif"
slide = pyvips.Image.new_from_file(path, page=0)
print(slide.width, slide.height)
>> 67456 84224

# load the 16x downsampled image from page 4 (2 ** 4 = 16)
slide = pyvips.Image.new_from_file(path, page=4)
print(slide.width, slide.height)
>> 4216 5264

# read an image region in as a PIL Image
x, y = 100, 200
level = 2
region_width = 500
region_height = 500
region = slide.crop(x, y, region_width, region_height)

# convert the PIL Image to a numpy array
import numpy

array = np.ndarray(
    buffer=region.write_to_memory(),
    dtype=np.uint8,
    shape=(region.height, region.width, region.bands)
)
# red, green, blue channels
print(array.shape)
>> (500, 500, 3)

# also look into pyvips.Region.fetch for faster region read
region = pyvips.Region.new(slide).fetch(x, y, region_width, region_height)

bands = 3
array = np.ndarray(
    buffer=region,
    dtype=np.uint8,
    shape=(region_height, region_width, bands)
)
print(array.shape)
>> (500, 500, 3)

If OpenSlide is installed on your machine, PyVips can also read in native WSI formats. In fact, this is how we converted WSIs to pyramidal TIFs for the competition. An example of the conversion is below:

import pyvips

native_path = "image_001.svs"  # path to the native WSI file
native_image = pyvips.Image.openslideload(native_path)
native_image.tiffsave(
    "image_001.tif",  # path to the output pyramidal TIF file
    compression="jpeg",
    Q=75,
    tile=True,
    tile_width=512,
    tile_height=512,
    pyramid=True,
)

Deep Zoom Viewer

OpenSlide Python provides support for Deep Zoom. This allows you to display images in a web browser without converting slides to the Deep Zoom format. Deep Zoom is useful if you want to efficiently view, pan around, and zoom whole slide images. Read more on the OpenSlide docs.

Cytomine

Cytomine allows you to display and explore native whole slide images and pyramidal TIF formats in a web browser. It also supports adding annotations and executing scripts from inside Cytomine or from any computing server using the dedicated Cytomine Python client. Cytomine can be installed locally or on any Linux server. The Cytomine GitHub repository includes examples of Python scripts demonstrating how to interact with your Cytomine instance, as well as examples of ready-to-use machine learning scripts (all S_ prefixed repos, such as S_CellDetect_Stardist_HE_ROI).

Tips and tricks

Here are a few tricks we discovered while working with the data:

  • A critical part of the challenge of this competition is how to efficiently process these enormous images. Due to the time limit on submissions, you will not have time to process every part of the image at the highest resolution available. Instead, you might consider methods to first predict which parts of the image are most important for an accurate diagnosis. The pyramidal TIF format contains multiple zoom levels that you can access with the page keyword argument to pyvips.Image.new_from_file. You might try a "multiscale" approach―detect the most important parts of the image from lower resolutions versions (pyvips.Image.new_from_file(path, page=5)) and then analyze just those parts at higher resolutions.
  • Use pyvips.Region.fetch instead of pyvips.Image.crop to read in small image regions. According to a PyVips developer and our own tests, pyvips.Region.new(image).fetch(...) is much faster than pyvips.Image.crop(...). (Requires libvips version 0.8.6 or greater.)
  • Avoid OpenSlide-based data loaders (pyvips.Image.openslideload and openslide.OpenSlide) in your machine learning pipelines. OpenSlide is no longer under active development, and several issues have arisen that have not been addressed. For example, we ran into errors reading certain SVS files (likely related to this open issue) in which parts of the image were blank. Even installing openslide-python with pip requires downgrading setuptools. We found that reading the pyramidal TIFs using PyVips was a reliable way to work with the images in our machine learning pipelines. (OpenSlide Python's Deep Zoom viewer is still helpful for manually browsing the slides.)

Additional reading and research

Here are a few papers and tutorials that talk about machine learning with WSI that you may find helpful:

On this page