DICOM Basics using Java - Segmentation Objects

Introduction

This is part of my series of articles on the DICOM standard. In this tutorial, we'll explore DICOM Segmentation objects, which store image segmentation results - masks identifying specific regions or structures in medical images. This is particularly relevant for AI/ML applications in radiology.

Segmentation objects enable storing deep learning model outputs, manual annotations, and quantitative analysis results in a standardized DICOM format.

Prerequisites

Before you begin, ensure you have the following:

• Java JDK installed (Java 8 or later)
• PixelMed Java DICOM Toolkit
• Understanding of DICOM multi-frame objects
• You can find all the code demonstrated in this tutorial on GitHub here

“The future of medicine is not in treatment, it’s in prediction.” ~ Eric Topol

Segmentation SOP Class

Element	Value
SOP Class UID	1.2.840.10008.5.1.4.1.1.66.4
Modality	SEG
Description	Segmentation Storage

The Theory Behind DICOM Segmentation

Segmentation objects represent the convergence of medical imaging and machine learning. As AI algorithms proliferate in radiology, standardized output formats become critical for clinical integration.

The AI Output Standardization Problem

Consider the challenge of integrating AI algorithms into clinical workflows:

• Algorithm A outputs a PNG mask with separate metadata file
• Algorithm B outputs a JSON file with polygon coordinates
• Algorithm C outputs a proprietary binary format

Each requires custom integration. DICOM Segmentation provides a universal output format that any compliant viewer can display, any PACS can store, and any downstream system can process.

The Multi-frame Design

Segmentation objects use DICOM's multi-frame architecture, where the segmentation mask for each source image slice becomes a frame in the SEG object. This design enables:

• 1:1 Mapping: Each SEG frame references a specific source image
• Multiple Segments: Different anatomical structures share the same multi-frame object
• Efficient Storage: Binary masks compress well with run-length encoding
• Spatial Alignment: Frame position matches source image position automatically

Binary vs. Fractional: A Design Decision

The choice between BINARY and FRACTIONAL segmentation reflects different use cases:

• BINARY: Each voxel is definitively inside or outside the structure. Appropriate for well-defined boundaries (liver edge, bone surface).
• FRACTIONAL PROBABILITY: Each voxel has a confidence score. Appropriate for AI model outputs where uncertainty should be preserved.
• FRACTIONAL OCCUPANCY: Each voxel indicates partial volume. Appropriate for structures smaller than voxel size.

Many AI models naturally output probability maps. Converting to binary requires choosing a threshold, which loses information. Fractional segmentation preserves the model's uncertainty for downstream decision-making.

Coded Semantics for Machine Reasoning

Each segment includes coded values from standard terminologies (SNOMED CT, SegmentedPropertyCategory/Type). This enables:

• Automatic identification of segment contents
• Cross-study comparison of same structure
• Decision support based on segment type
• Consistent display colors across viewers

A "liver" segment coded with SNOMED CT code T-62000 can be automatically recognized by any system, regardless of the algorithm that created it.

Common Use Cases

• AI/ML Model Outputs: Tumor detection, organ segmentation, lesion characterization
• Manual Annotations: Radiologist annotations, ground truth labels for AI training
• Quantitative Analysis: Volume measurements, longitudinal tracking
• Treatment Planning: Auto-contouring for radiation therapy, surgical planning

Segmentation IOD Structure

System.out.println("=== DICOM Segmentation Object Demo ===\n");

System.out.println("Key Modules:");
System.out.println("  Patient Module - Patient demographics");
System.out.println("  General Study Module - Study information");
System.out.println("  Segmentation Series Module:");
System.out.println("    Modality = SEG");
System.out.println("  Multi-frame Dimension Module:");
System.out.println("    Dimension Organization Sequence");
System.out.println("    Dimension Index Sequence");
System.out.println("  Segmentation Image Module:");
System.out.println("    Segmentation Type = BINARY or FRACTIONAL");
System.out.println("    Segment Sequence (defines each segment)");
System.out.println("  Multi-frame Functional Groups Module:");
System.out.println("    Per-Frame Functional Groups Sequence");
System.out.println("  Pixel Data Module:");
System.out.println("    Pixel Data (the actual mask data)");

Segment Sequence Example

Segment Sequence (0062,0002):

  Segment 1:
    Segment Number: 1
    Segment Label: Liver
    Segment Description: Liver parenchyma segmentation
    Segment Algorithm Type: AUTOMATIC
    Segment Algorithm Name: AI Liver Seg v2.0
    Segmented Property Category Code Sequence:
      Code Value: T-D0050
      Coding Scheme: SRT
      Code Meaning: Tissue
    Segmented Property Type Code Sequence:
      Code Value: T-62000
      Coding Scheme: SRT
      Code Meaning: Liver
    Recommended Display CIELab Value: [39330, 34471, 26163] (brownish)

  Segment 2:
    Segment Number: 2
    Segment Label: Liver Tumor
    Segment Description: Hepatic lesion
    Segment Algorithm Type: SEMIAUTOMATIC
    Segmented Property Category Code Sequence:
      Code Value: M-80003
      Coding Scheme: SRT
      Code Meaning: Neoplasm
    Segmented Property Type Code Sequence:
      Code Value: M-80006
      Coding Scheme: SRT
      Code Meaning: Malignant Neoplasm
    Recommended Display CIELab Value: [62431, 54116, 45216] (reddish)

Segmentation Types

BINARY Segmentation:

• Each pixel is either 0 (outside) or 1 (inside)
• Bits Allocated = 1
• Most common type
• Efficient storage
• Example: Organ boundaries, tumor contours

FRACTIONAL Segmentation:

• Each pixel has a probability/fraction (0.0 to 1.0)
• Bits Allocated = 8 (stored as 0-255)
• Segmentation Fractional Type:
  • PROBABILITY: Values represent confidence
  • OCCUPANCY: Values represent partial occupancy
• Used for probability maps from AI models

Segment Algorithm Types

Type	Description
AUTOMATIC	Fully automated segmentation (AI/ML)
SEMIAUTOMATIC	Automated with user edits
MANUAL	Hand-drawn segmentation

Key Attributes

Tag	Name	Description
(0062,0001)	Segmentation Type	BINARY or FRACTIONAL
(0062,0002)	Segment Sequence	Defines each segment
(0062,0004)	Segment Number	Unique segment identifier
(0062,0005)	Segment Label	User-readable name
(0062,0006)	Segment Description	Detailed description
(0062,0008)	Segment Algorithm Type	AUTOMATIC, SEMIAUTOMATIC, MANUAL
(0062,0003)	Segmented Property Category	Category code (e.g., Tissue)
(0062,000F)	Segmented Property Type	Type code (e.g., Liver)

Integration with PACS

Segmentation objects integrate with existing PACS workflows:

• Stored alongside original images in PACS
• Retrieved and displayed as overlays
• Referenced in Structured Reports
• Used in RT Structure Set generation
• Queried via C-FIND (Modality = SEG)

Tools for Creating Segmentations

Tool	Description
dcmqi	QIICR DICOM for Quantitative Imaging
pydicom-seg	Python library for DICOM SEG
3D Slicer	Open-source medical imaging platform
ITK-SNAP	Interactive segmentation tool
PixelMed	Manual construction via Java API

Useful Resources

• dcmqi - DICOM for Quantitative Imaging
• dcmqi Documentation
• DICOM PS3.3 Section A.51 (Segmentation IOD)

Conclusion

DICOM Segmentation objects provide a standardized way to store and exchange image segmentation results. With the growth of AI/ML in medical imaging, understanding segmentation objects is essential for integrating automated analysis results into clinical workflows.

Whether storing deep learning model outputs, manual annotations, or quantitative measurements, DICOM segmentation enables interoperable storage and display of region-of-interest data across healthcare systems. In the next tutorial in this series, I will cover DICOM Radiation Therapy (RT) objects used in radiation oncology. See you then!