Windows ML and DirectML: Understanding the Relationship

This article explores the technical relationship between the Windows Machine Learning (Windows ML) API and the DirectML engine within the DirectX ecosystem. It breaks down the layered architecture of Microsoft’s machine learning stack, detailing how high-level model evaluation in Windows ML translates into low-level, hardware-accelerated compute commands via DirectML across diverse silicon.

The Architectural Hierarchy

Windows ML and DirectML are not competing technologies; they represent two distinct layers of Microsoft’s AI inferencing stack on Windows. Windows ML sits at the top as a high-level, developer-friendly API, while DirectML serves as the low-level, high-performance hardware acceleration library built directly on top of DirectX 12.

+-------------------------------------------------------+
|                 Application Layer                     |
+-------------------------------------------------------+
|              Windows ML (WinRT / C++ / C#)            |
+-------------------------------------------------------+
|                    ONNX Runtime                       |
+-------------------------------------------------------+
|             DirectML Execution Provider               |
+-------------------------------------------------------+
|                DirectML (DirectX 12)                  |
+-------------------------------------------------------+
|       GPU / NPU Hardware (AMD, Intel, NVIDIA, Qualcomm)|
+-------------------------------------------------------+

Windows ML: The High-Level Runtime

Windows ML is a Windows Runtime (WinRT) component designed to simplify adding trained machine learning models to Windows applications. It accepts models formatted in the Open Neural Network Exchange (ONNX) standard.

Developers interact with Windows ML using simple abstractions:

  • LearningModel: Represents the loaded ONNX model.
  • LearningModelSession: Handles the execution environment and binds hardware targets (CPU, GPU).
  • LearningModelBinding: Binds input and output variables such as tensors or image frames.

Windows ML abstracts away memory management, operator mapping, and graphics pipeline synchronization. Under the hood, Windows ML packages and integrates the ONNX Runtime engine directly into the Windows operating system.

DirectML: The DirectX 12 Compute Engine

DirectML is a low-level, high-performance DirectX 12 library specifically designed for machine learning. It exposes an extensive collection of optimized mathematical operators (such as convolutions, activations, and matrix multiplications) implemented as compute shaders.

Because DirectML is fundamentally part of DirectX:

  • Hardware Agnostic: It runs across any DirectX 12-compatible device, including GPUs and NPUs from AMD, Intel, NVIDIA, and Qualcomm.
  • Low-Level Control: It gives developers explicit control over resource allocation, descriptor heaps, command lists, and synchronization fences.
  • Low Latency: It eliminates framework overhead by executing directly on the graphics driver compute queue.

How Windows ML Uses DirectML

When an application evaluates a model using Windows ML, the framework coordinates with DirectML through the following sequence:

  1. Model Parsing: Windows ML loads the ONNX graph through the integrated ONNX Runtime.
  2. Provider Selection: If the developer requests GPU acceleration (or relies on default hardware selection), the ONNX Runtime delegates compute nodes to its DirectML Execution Provider.
  3. Graph Translation: The DirectML Execution Provider maps ONNX operators to corresponding DirectML operator initializers.
  4. DirectX 12 Execution: DirectML compiles these operators into GPU compute shaders, constructs DirectX 12 command lists, and submits them to the driver's compute queue.
  5. Output Delivery: Results are retrieved from the hardware buffer and returned to the application through Windows ML’s output bindings.

DirectML vs. Windows ML: Choosing the Right Layer

Developers choose between the two layers based on control requirements:

  • Use Windows ML if your goal is rapid integration of standard ONNX models into desktop, WinUI, or UWP applications without dealing with graphics APIs or manual GPU resource management.
  • Use DirectML directly if you are integrating inferencing into an existing DirectX 12 engine (such as a game engine), need custom GPU operator pipelines, or require precise control over memory allocations and GPU command scheduling.