Cell Behavior Video Classification Challenge

Quick links

• Dataset (phase1 and phase 2)
• Code (GitHub repository)

Leaderboard

• View Challenge leaderboard
• View Post-challenge leaderboard
• Submit your results *

🔍 Introduction and Goal

The Cell Behavior Video Classification Challenge (CBVCC) is designed to develop or adapt computer vision methods for classifying videos capturing cell behavior through Intravital Microscopy (IVM). IVM is a powerful imaging technique that allows for non-invasive visualization of biological processes in living animals. Platforms such as two-photon microscopes exploit multiple low-energy photons to deliver high-resolution, three-dimensional videos depicting tissues and cells deep within the body. IVM has been used to visualize a wide range of biological processes, including immune responses, cancer development, and neurovascular function.

The primary goal of the CBVCC challenge is to create models that can accurately classify videos based on the movement patterns of cells. Specifically, the models should be capable of:

• Identifying videos where cells exhibit sudden changes in their direction of movement.
• Distinguishing these from videos where cells show consistent, linear movement, stationary cells, or videos containing only background.

The CBVCC challenge aims to provide a platform for researchers to develop innovative methods for classifying IVM videos, potentially leading to new insights into biological processes.

👩‍🔬👨‍🔬 The CBVCC Challenge

The CBVCC challenge is open to researchers worldwide. Participants are tasked with developing computational models to classify IVM videos. The models will be evaluated based on their accuracy in classifying videos.

The challenge is divided into two phases:

• Phase 1: Participants will be provided with a training dataset and a test dataset of IVM images and labels to develop and evaluate their models.
• Phase 2: Participants will submit their results obtained on a distinct test dataset of IVM images. Models will be ranked based on their performance on this test dataset.

The final results will be determined based on the performance of the Phase 2 submissions.

🗓 The Challenge Timeline

• September 15th, 2024 – November 15th, 2024: Registration period. Participants can register at this link.
• November 15th, 2024: Training dataset released; validation leaderboard becomes available.
• December 13th, 2024: Unannotated test dataset released.
• December 20th, 2024: Challenge closes. Participants must submit predictions on the test set, an abstract describing their methodology, and code or a Docker container to replicate results.
• January 7th, 2025: Results and winners announced.

The CBVCC dataset comprises 2D video patches extracted from IVM videos capturing the behavior of regulatory T cells in the abdominal flank skin of mice undergoing a contact sensitivity response to the sensitizing hapten, oxazolone. Videos are acquired either 24 or 48 hours after the initial skin challenge, with each sequence lasting 30 minutes and images captured at one-minute intervals, resulting in 31 acquisitions per video.

The dataset includes:

• Class 1: Video patches where cells suddenly change their direction of movement.
• Class 0: Video patches without sudden changes in cell direction, including linear movement, stationary cells, or background-only videos.

The dataset consists of 300 2D video patches (180 class 0 and 120 class 1) extracted from 48 different videos. Each patch is a 2D projection along the z-axis of a 3D video sequence, adjusted to a common contrast range to enhance visibility. All videos have been preprocessed to ensure a uniform pixel size of 0.8 µm and are saved as RGB .avi files.

Dataset splits:

• Training set: 210 labeled video patches (70%) – Download
• Phase 1 test set: 30 labeled video patches (10%) – Download
• Phase 2 test set: 60 unlabeled video patches (20%) – Download

Each subset includes video patches extracted from different and independent IVM videos. In class 1, the change of direction occurs at the midpoint and center of the video patches.

Additional files (not provided during the challenge):

• Ground truth labels for test set (phase 2) – Download
• Manual tracking - Download

📊 Evaluation

Models will be evaluated using the following metrics:

• Area Under the ROC Curve (AUC)
• Precision
• Recall
• Balanced Accuracy

The final score is calculated as:

Score = 0.4 × AUC + 0.2 × (Precision + Recall + Balanced Accuracy)

The challenge is now closed.
Results can be submitted to the post.challenge leaderboards. To do so submit the results of your method in a CSV file indicating the video id and the cass-1 probability [from 0 to 1].
(e.g.
02_12.avi,0.48
02_4.avi,0.65
02_3.avi,0.84
...
)
An example can be found here: https://www.dp-lab.info/cbvcc/sample_file.csv

Link to submit your results on the validation set (phase 1): https://www.dp-lab.info/cbvcc/data/submitpost1.php
Link to submit your results on the test set (phase 2): https://www.dp-lab.info/cbvcc/data/submitpost2.php

Leaderboard - Validation set (phase 1)

Leaderboard - Test set (phase 2)

Results on Test set (phase 2)

Results on Validations et (phase 1)

Ensure to reload the page to visualize the latest results.