DICOM Basics using .NET and C# - Segmentation Objects

Introduction

This is part of my series of articles on the DICOM standard. In this tutorial, we'll explore DICOM Segmentation objects, which provide a standardized way to store labeled regions (segments) that identify anatomical structures or lesions in medical images.

Segmentation objects are increasingly important for AI/ML applications, surgical planning, and quantitative analysis. Unlike RT Structure Sets which store contours, Segmentation objects store voxel-level labels.

Prerequisites

Before you begin, ensure you have the following:

• A .NET development environment (Visual Studio or Visual Studio Code)
• The Fellow Oak DICOM library (fo-dicom) installed via NuGet
• Basic understanding of DICOM concepts from previous tutorials
• You can find all the code demonstrated in this tutorial on GitHub here

“The whole is greater than the sum of its parts.” ~ Aristotle

The Theory Behind Segmentation Objects

As AI/ML transforms medical imaging, a critical question emerges: how do you store algorithm outputs? A neural network that segments liver tumors produces a mask identifying which voxels are tumor. Without a standard format, every vendor invents proprietary representations, making results non-interoperable. DICOM Segmentation provides that standardized output format for AI algorithms, enabling results from different systems to be stored, viewed, and processed uniformly.

The multi-frame architecture is central to segmentation design. Rather than storing one object per slice, a segmentation stores all data in a single multi-frame image. Frame 1 might be Segment 1 (liver) on slice 1, Frame 2 is Segment 1 on slice 2, and so on. The Per-Frame Functional Groups sequence describes what each frame represents. This compact representation is efficient for storage and ensures all segmentation data travels as one unit.

The choice between BINARY and FRACTIONAL types reflects different use cases. Binary segmentation (each voxel is IN or OUT) suits discrete anatomical boundaries: is this voxel liver or not? Fractional segmentation (probability 0.0 to 1.0) suits uncertainty or partial volume effects: AI confidence scores, or tissue that's partially in the segment. Fractional preserves information that binary thresholding would discard.

The coded semantics in Segment Sequence provide meaning to segments. Segment #1 isn't just "the first segment" - it's explicitly coded as (T-62000, SRT, "Liver") using standard terminology. This enables automated processing: a liver volumetry algorithm can find the liver segment by code rather than assuming segment ordering. The Segmented Property Category further classifies (Tissue, Organ, Lesion, etc.).

Compared to RT Structure Sets (contours), segmentation provides voxel-level precision at the cost of larger file sizes. Contours are memory-efficient but require interpolation between slices. For AI outputs and quantitative analysis where precise boundaries matter, voxel-based segmentation is often preferred. Many workflows convert between formats as needed.

Understanding Segmentation Objects

DICOM Segmentation is defined by SOP Class 1.2.840.10008.5.1.4.1.1.66.4:

Aspect	Description
SOP Class	Segmentation Storage
UID	1.2.840.10008.5.1.4.1.1.66.4
Modality	SEG
Content	Voxel-level segment labels
Types	Binary or Fractional

Segmentation Types

DICOM supports two segmentation types:

using System;
using System.Diagnostics;
using FellowOakDicom;

namespace DicomSegmentation
{
    public class Program
    {
        public static void Main(string[] args)
        {
            LogToDebugConsole("=== DICOM Segmentation Demo ===");
            LogToDebugConsole("");

            DemonstrateSegmentationTypes();
            DemonstrateSegmentationStructure();
            DemonstrateSegmentSequence();
        }

        private static void DemonstrateSegmentationTypes()
        {
            LogToDebugConsole("--- Segmentation Types ---");
            LogToDebugConsole("");

            LogToDebugConsole("1. BINARY Segmentation:");
            LogToDebugConsole("   - Each voxel is either IN (1) or OUT (0)");
            LogToDebugConsole("   - Stored as packed bits (8 pixels per byte)");
            LogToDebugConsole("   - Most common type");
            LogToDebugConsole("   - Use for discrete anatomical boundaries");
            LogToDebugConsole("");

            LogToDebugConsole("2. FRACTIONAL Segmentation:");
            LogToDebugConsole("   - Each voxel has a probability (0.0 to 1.0)");
            LogToDebugConsole("   - Stored as 8-bit values (0-255)");
            LogToDebugConsole("   - Use for uncertainty or partial volume effects");
            LogToDebugConsole("   - Common for AI/ML outputs");
            LogToDebugConsole("");

            LogToDebugConsole("Fractional Types:");
            LogToDebugConsole("   PROBABILITY - likelihood of segment membership");
            LogToDebugConsole("   OCCUPANCY - fraction of voxel occupied");
        }

        private static void LogToDebugConsole(string message)
        {
            Debug.WriteLine(message);
        }
    }
}

Key Segmentation Attributes

Important attributes in a Segmentation object:

Tag	Name	Description
(0062,0001)	Segmentation Type	BINARY or FRACTIONAL
(0062,0002)	Segment Sequence	Defines each segment
(0062,0004)	Segment Number	Unique segment identifier
(0062,0005)	Segment Label	Human-readable name
(0062,0006)	Segment Description	Detailed description
(0062,0008)	Segment Algorithm Type	AUTOMATIC, SEMIAUTOMATIC, MANUAL
(0062,0003)	Segmented Property Category	Coded anatomical category
(0062,000D)	Recommended Display CIELab Value	Color for display

Segmentation Structure

A Segmentation object contains multiple components:

private static void DemonstrateSegmentationStructure()
{
    LogToDebugConsole("--- Segmentation Structure ---");
    LogToDebugConsole("");

    LogToDebugConsole("Required Modules:");
    LogToDebugConsole("  - Patient Module");
    LogToDebugConsole("  - General Study Module");
    LogToDebugConsole("  - General Series Module (Modality = SEG)");
    LogToDebugConsole("  - Frame of Reference Module");
    LogToDebugConsole("  - General Equipment Module");
    LogToDebugConsole("  - Enhanced General Equipment Module");
    LogToDebugConsole("  - General Image Module");
    LogToDebugConsole("  - Image Pixel Module");
    LogToDebugConsole("  - Segmentation Image Module");
    LogToDebugConsole("  - Multi-frame Functional Groups Module");
    LogToDebugConsole("  - Multi-frame Dimension Module");
    LogToDebugConsole("");

    LogToDebugConsole("Pixel Data Encoding:");
    LogToDebugConsole("  - Multi-frame format (one frame per segment per slice)");
    LogToDebugConsole("  - Binary: 1-bit per pixel, packed 8 per byte");
    LogToDebugConsole("  - Fractional: 8-bit per pixel (0-255)");
    LogToDebugConsole("");

    LogToDebugConsole("Frame Organization:");
    LogToDebugConsole("  Frame 1: Segment 1, Slice 1");
    LogToDebugConsole("  Frame 2: Segment 1, Slice 2");
    LogToDebugConsole("  ...");
    LogToDebugConsole("  Frame N: Segment 2, Slice 1");
    LogToDebugConsole("  (varies by implementation)");
}

Segment Sequence

The Segment Sequence defines each labeled region:

private static void DemonstrateSegmentSequence()
{
    LogToDebugConsole("--- Segment Sequence (0062,0002) ---");
    LogToDebugConsole("");

    LogToDebugConsole("Example Segment Sequence:");
    LogToDebugConsole("");

    LogToDebugConsole("Segment 1:");
    LogToDebugConsole("  (0062,0004) Segment Number: 1");
    LogToDebugConsole("  (0062,0005) Segment Label: \"Liver\"");
    LogToDebugConsole("  (0062,0008) Algorithm Type: AUTOMATIC");
    LogToDebugConsole("  (0062,0009) Algorithm Name: \"AI Liver Segmentation v2.1\"");
    LogToDebugConsole("");
    LogToDebugConsole("  Segmented Property Category Code Sequence:");
    LogToDebugConsole("    Code Value: T-D0050");
    LogToDebugConsole("    Coding Scheme: SRT");
    LogToDebugConsole("    Code Meaning: \"Tissue\"");
    LogToDebugConsole("");
    LogToDebugConsole("  Segmented Property Type Code Sequence:");
    LogToDebugConsole("    Code Value: T-62000");
    LogToDebugConsole("    Coding Scheme: SRT");
    LogToDebugConsole("    Code Meaning: \"Liver\"");
    LogToDebugConsole("");
    LogToDebugConsole("  Recommended Display CIELab Value: 39077\\54285\\33026 (brown)");
    LogToDebugConsole("");

    LogToDebugConsole("Segment 2:");
    LogToDebugConsole("  (0062,0004) Segment Number: 2");
    LogToDebugConsole("  (0062,0005) Segment Label: \"Tumor\"");
    LogToDebugConsole("  (0062,0008) Algorithm Type: SEMIAUTOMATIC");
    LogToDebugConsole("  Segmented Property Type: (M-80003) \"Neoplasm\"");
    LogToDebugConsole("  Recommended Display CIELab Value: (red)");
}

Functional Groups

Segmentation uses multi-frame functional groups to describe each frame:

private static void DemonstrateFunctionalGroups()
{
    LogToDebugConsole("--- Per-Frame Functional Groups ---");
    LogToDebugConsole("");

    LogToDebugConsole("Derivation Image Functional Group:");
    LogToDebugConsole("  - References source images (CT/MR)");
    LogToDebugConsole("  - Links segment to original image slices");
    LogToDebugConsole("");

    LogToDebugConsole("Frame Content Functional Group:");
    LogToDebugConsole("  - Dimension Index Values");
    LogToDebugConsole("  - Stack ID");
    LogToDebugConsole("");

    LogToDebugConsole("Plane Position (Patient) Functional Group:");
    LogToDebugConsole("  - Image Position (Patient)");
    LogToDebugConsole("  - Locates each frame in 3D space");
    LogToDebugConsole("");

    LogToDebugConsole("Segment Identification Functional Group:");
    LogToDebugConsole("  - Referenced Segment Number");
    LogToDebugConsole("  - Links frame to Segment Sequence item");
}

Reading Segmentation Objects

Here's how to read a Segmentation object:

public static void ReadSegmentation(string filePath)
{
    var file = DicomFile.Open(filePath);
    var dataset = file.Dataset;

    // Verify SOP Class
    var sopClass = dataset.GetSingleValueOrDefault(DicomTag.SOPClassUID, "");
    if (sopClass != DicomUID.SegmentationStorage.UID)
    {
        LogToDebugConsole("Not a Segmentation object");
        return;
    }

    LogToDebugConsole($"Modality: {dataset.GetSingleValueOrDefault(DicomTag.Modality, "")}");

    // Get segmentation type
    var segType = dataset.GetSingleValueOrDefault(DicomTag.SegmentationType, "");
    LogToDebugConsole($"Segmentation Type: {segType}");

    // Get number of frames
    var numFrames = dataset.GetSingleValueOrDefault(DicomTag.NumberOfFrames, 0);
    LogToDebugConsole($"Number of Frames: {numFrames}");

    // Read Segment Sequence
    var segmentSeq = dataset.GetSequence(DicomTag.SegmentSequence);
    if (segmentSeq != null)
    {
        LogToDebugConsole($"Number of Segments: {segmentSeq.Items.Count}");

        foreach (var item in segmentSeq.Items)
        {
            var segNum = item.GetSingleValueOrDefault(DicomTag.SegmentNumber, 0);
            var segLabel = item.GetSingleValueOrDefault(DicomTag.SegmentLabel, "");
            var algType = item.GetSingleValueOrDefault(DicomTag.SegmentAlgorithmType, "");

            LogToDebugConsole($"  Segment {segNum}: {segLabel} ({algType})");
        }
    }
}

Common Use Cases

• AI/ML Outputs: Store neural network segmentation results
• Tumor Volumetry: Calculate tumor volumes from segmented regions
• Surgical Planning: Identify critical structures for surgery
• Radiation Therapy: Alternative to RT Structure Set for some workflows
• Research: Standardized format for sharing annotated datasets

Segmentation vs RT Structure Set

Aspect	Segmentation	RT Structure Set
Storage	Voxel labels	Contour points
Format	Multi-frame image	Sequences
Precision	Voxel-level	Contour interpolation
File Size	Larger	Smaller
Primary Use	Quantitative analysis, AI	Radiation therapy
Modality	SEG	RTSTRUCT

Best Practices

• Use coded concepts: Standard codes (SNOMED, SCT) for anatomy
• Include algorithm info: Document how segments were created
• Set display colors: Provide recommended CIELab colors
• Reference source images: Link to original imaging data
• Choose appropriate type: Binary for discrete regions, Fractional for probabilities

Conclusion

DICOM Segmentation objects provide a standardized way to store and share labeled anatomical regions. They are increasingly important for AI/ML applications that produce segmentation masks and for quantitative image analysis workflows.

Understanding Segmentation objects is essential for anyone working with medical image analysis, computer-aided detection, or AI-based diagnostic tools. The voxel-level storage enables precise volumetric measurements and seamless integration with analysis pipelines.

Please check out the next tutorial in this series where we cover DICOM Waveforms (ECG, EEG).