Kernel

Linux 7.2 Seeds "Blackwell-Next": A Deep Dive into the nvgrace-gpu VFIO CXL DVSEC Change

Linux 7.2 Seeds "Blackwell-Next": A Deep Dive into the nvgrace-gpu VFIO CXL DVSEC Change

Linux 7.2’s VFIO pull request dropped a commit with a codename I hadn’t seen before: Blackwell-Next. A Phoronix post brought this to my attention - Linux 7.2 Begins Making Preparations For NVIDIA “Blackwell-Next” - which, on the face of it looks like a minor prep patch. It is — but it’s also a clean window into where NVIDIA is taking its CPU-coherent GPU stack, how CXL is quietly becoming the standard signaling interface for next-generation accelerators, and what that means if you’re building infrastructure or tooling on top of these platforms.

Read More
Linux Kernel v7.1 is Released: This is What's New for Compute Express Link (CXL)

Linux Kernel v7.1 is Released: This is What's New for Compute Express Link (CXL)

The Linux Kernel v7.1 release brings several improvements and additions related to Compute Express Link (CXL) technology.

Release Highlights

Linux Kernel v7.1 includes 47 commits to the CXL and DAX subsystems:

CategoryCommits
New Features & Hardware1
Bug Fixes5
Refactoring & Cleanup5
Testing1
Other35

The v7.1 CXL/DAX cycle is defined by three interlocking themes: laying the groundwork for Type 2 accelerator support, hardening the DAX/HMEM subsystem against a cluster of correctness bugs, and a focused refactoring of the region layer that splits a monolithic file into purpose-specific translation units. None of these is a headline splash feature on its own, but together they represent the kind of steady, unglamorous investment that makes the subsystem reliable enough to build production systems on.

Read More
Linux Kernel v7.0 is Released: This is What's New for Compute Express Link (CXL)

Linux Kernel v7.0 is Released: This is What's New for Compute Express Link (CXL)

The Linux Kernel v7.0 release brings several improvements and additions related to Compute Express Link (CXL) technology.

Release Highlights

Linux Kernel v7.0 includes 73 commits to the CXL and DAX subsystems:

CategoryCommits
New Features & Hardware3
Bug Fixes11
Refactoring & Cleanup9
Testing2
Other48

Linux v7.0 brings focused but meaningful progress to the CXL/DAX subsystem, with the headline work centered on platform-specific address translation. The cxl/atl subsystem gains AMD Zen5 support through the ACPI Platform Runtime Mechanism Table (PRMT), enabling hardware-assisted Host Physical Address (HPA) to System Physical Address (SPA) translation on AMD’s latest server platforms. This required scaffolding across several layers: EFI runtime services preparation in cxl/acpi, new translation callback hooks, decoder locking for address translation paths, and explicit disabling of these handlers when Normalized Addressing is active — a sign the translation infrastructure is maturing toward multi-vendor, multi-mode support.

Read More
Linux Kernel v6.19 is Released: This is What's New for Compute Express Link (CXL)

Linux Kernel v6.19 is Released: This is What's New for Compute Express Link (CXL)

The Linux Kernel v6.19 release brings several improvements and additions related to Compute Express Link (CXL) technology.

Release Highlights

Linux Kernel v6.19 includes 31 commits to the CXL and DAX subsystems:

CategoryCommits
New Features & Hardware1
Bug Fixes4
Refactoring & Cleanup5
Testing1
Documentation2
Other18

Linux v6.19 is a measured release for the CXL/DAX subsystem — 31 commits spread across correctness fixes, code hardening, and targeted new functionality. The headline addition is extended linear cache (ELC) region support: regions can now be flagged to indicate they carry an ELC mapping, a prerequisite for properly managing CXL memory that participates in CPU-side cache hierarchies. Alongside the feature itself, the release includes adjustments to how ELC failures are reported through cxl_acpi, and the HBIW platform-data guard that was accidentally dropped has been restored.

Read More
Linux Kernel v6.18 is Released: This is What's New for Compute Express Link (CXL)

Linux Kernel v6.18 is Released: This is What's New for Compute Express Link (CXL)

The Linux Kernel v6.18 release brings several improvements and additions related to Compute Express Link (CXL) technology.

Release Highlights

Linux Kernel v6.18 includes 32 commits to the CXL and DAX subsystems:

CategoryCommits
New Features & Hardware1
Bug Fixes4
Refactoring & Cleanup5
Testing2
Other20

The v6.18 kernel cycle for CXL/DAX is defined by two architectural threads running in parallel: hardening the address-translation stack and untangling port initialization from topology discovery. On the translation side, the new SPA-to-DPA region mapping infrastructure lands alongside a dedicated root-decoder ops structure that formalizes how the host physical address space is projected into CXL’s device physical address space — including XOR-interleaving math that was previously implicit. These foundations make region geometry computable and auditable in ways that earlier releases left to convention.

Read More
Linux Kernel v6.17 is Released: This is What's New for Compute Express Link (CXL)

Linux Kernel v6.17 is Released: This is What's New for Compute Express Link (CXL)

The Linux Kernel v6.17 release brings several improvements and additions related to Compute Express Link (CXL) technology.

Release Highlights

Linux Kernel v6.17 includes 32 commits to the CXL and DAX subsystems:

CategoryCommits
New Features & Hardware1
Bug Fixes6
Refactoring & Cleanup3
Other22

The v6.17 cycle for CXL and DAX is a consolidation release rather than a feature-heavy one, with 32 commits that reflect the subsystem maturing around correctness, specification compliance, and architectural hygiene. The most visible theme is alignment with CXL specification revision 3.2: the Common Event Record has been updated to match the new spec, the Memory Sparing Event Record gains kernel tracing support for the first time, and additional validity checks land for corrected volatile memory error (CVME) counts in both DRAM and General Media Event Records. This work strengthens the kernel’s ability to correctly interpret and surface CXL RAS events to userspace tooling and monitoring infrastructure.

Read More
Linux Kernel v6.16 is Released: This is What's New for Compute Express Link (CXL)

Linux Kernel v6.16 is Released: This is What's New for Compute Express Link (CXL)

The Linux Kernel v6.16 release brings several improvements and additions related to Compute Express Link (CXL) technology.

Release Highlights

Linux Kernel v6.16 includes 37 commits to the CXL and DAX subsystems:

CategoryCommits
New Features & Hardware2
Bug Fixes6
Refactoring & Cleanup8
Documentation3
Other18

The Linux v6.16 kernel cycle is dominated by one clear theme: hardening CXL memory device reliability and serviceability through the EDAC subsystem. Four new control features land in this release — patrol scrub, Error Check Scrub (ECS), soft Post Package Repair (PPR), and memory sparing — each exposing a distinct class of CXL 3.0 memory maintenance operations to userspace through a consistent sysfs interface. Alongside these, support for the PERFORM_MAINTENANCE command provides the underlying mechanism that drives scrub and repair operations on compliant devices. Taken together, this work moves CXL from a device class that Linux can merely enumerate and map to one where the kernel actively participates in proactive memory health management.

Read More
Linux Kernel v6.15 is Released: This is What's New for Compute Express Link (CXL)

Linux Kernel v6.15 is Released: This is What's New for Compute Express Link (CXL)

The Linux Kernel v6.15 release brings several improvements and additions related to Compute Express Link (CXL) technology.

Release Highlights

Linux Kernel v6.15 includes 55 commits to the CXL and DAX subsystems:

CategoryCommits
New Features & Hardware6
Bug Fixes4
Performance1
Refactoring & Cleanup9
Other35

The Linux v6.15 kernel marks a meaningful expansion of CXL’s userspace interface story. The headline addition is FWCTL support: CXL devices can now expose get-feature and set-feature mailbox commands to userspace through the fwctl subsystem, giving operators and tooling a standardized RPC path to query and configure device-specific feature registers without requiring bespoke kernel drivers for each capability. This is the groundwork that enables feature negotiation at the management layer — expect CXL tooling to start consuming these interfaces quickly.

Read More
Linux Kernel 6.14 is Released: This is What's New for Compute Express Link (CXL)

Linux Kernel 6.14 is Released: This is What's New for Compute Express Link (CXL)

The Linux Kernel 6.14 release brings several improvements and additions related to Compute Express Link (CXL) technology.

Here is the detailed list of all commits merged into the 6.14 Kernel for CXL and DAX. This list was generated by the Linux Kernel CXL Feature Tracker .

Read More
Linux Kernel v6.14 is Released: This is What's New for Compute Express Link (CXL)

Linux Kernel v6.14 is Released: This is What's New for Compute Express Link (CXL)

The Linux Kernel v6.14 release brings several improvements and additions related to Compute Express Link (CXL) technology.

Release Highlights

Linux Kernel v6.14 includes 13 commits to the CXL and DAX subsystems:

CategoryCommits
Bug Fixes1
Refactoring & Cleanup2
Other10

The dominant story in v6.14’s CXL changes is alignment with CXL specification revision 3.1 in the event subsystem. Five event record types — Common, General Media, DRAM, Memory Module, and Component Identifier — were updated to match the latest spec. These records are how CXL devices surface hardware faults, media errors, and performance anomalies to the host, so keeping them in sync with the specification is critical for accurate error classification and interoperability with newer hardware that implements the 3.1 format changes.

Read More
Linux Kernel 6.13 is Released: This is What's New for Compute Express Link (CXL)

Linux Kernel 6.13 is Released: This is What's New for Compute Express Link (CXL)

The Linux Kernel 6.13 release brings several improvements and additions related to Compute Express Link (CXL) technology.

Here is the detailed list of all commits merged into the 6.13 Kernel for CXL and DAX. This list was generated by the Linux Kernel CXL Feature Tracker .

CXL related changes from Kernel v6.12 to v6.13:

Read More
Linux Kernel 6.10 is Released: This is What's New for Compute Express Link (CXL)

Linux Kernel 6.10 is Released: This is What's New for Compute Express Link (CXL)

The Linux Kernel 6.10 release brings several improvements and additions related to Compute Express Link (CXL) technology.

Here is the detailed list of all commits merged into the 6.10 Kernel for CXL and DAX. This list was generated by the Linux Kernel CXL Feature Tracker .

Read More
Linux Kernel 6.9 is Released: This is What's New for Compute Express Link (CXL)

Linux Kernel 6.9 is Released: This is What's New for Compute Express Link (CXL)

The Linux Kernel 6.9 release brings several improvements and additions related to Compute Express Link (CXL) technology.

New Features

Here is a list of new features for CXL:

Here is the detailed list of all commits merged into the 6.9 Kernel for CXL and DAX. This list was generated by the Linux Kernel CXL Feature Tracker .

Read More
Linux Kernel CXL Feature Tracker

Linux Kernel CXL Feature Tracker

I’m always watching the Linux Kernel for new and exciting features that are merged for Compute Express Link (CXL). There’s some great notes from the monthly developer meetup here , but the devil is always in the details, and not every commit is discussed in the meeting. So I wrote a simple Python script, called cxl_feature_tracker.py that looks in all commits to the Linus Torvalds Linux Kernel GitHub repository , and extracts any that mention “CXL” or “DAX”, or that make changes to the drivers/cxl or drivers/dax directories. The output is a very long list, but it has some gems amongst the list of fixes.

Read More
Using Linux Kernel Tiering with Compute Express Link (CXL) Memory

Using Linux Kernel Tiering with Compute Express Link (CXL) Memory

In this blog post, we will walk through the process of enabling the Linux Kernel Transparent Page Placement (TPP) feature with CXL memory mapped as NUMA nodes using the system-ram namespace. This feature allows the kernel to automatically place pages in different types of memory based on their usage patterns.

Prerequisites

This guide assumes that you are using a Fedora 36 system with Kernel 5.19.13, and that your system has a Samsung CXL device installed. You can confirm the presence of the CXL device with the following command:

Read More

How To Install a Mainline Linux Kernel in Ubuntu

Note: This article was updated on Thursday, July 31st, 2025 and will work with newer Ubuntu releases.

By default, Ubuntu systems run with the Ubuntu kernels provided by the Ubuntu repositories. To get unmodified upstream kernels that have new features or to confirm that upstream has fixed a specific issue, we often need to install the mainline Kernel. The mainline kernel is the most recent version of the Linux kernel released by the Linux Kernel Organization. It undergoes several stages of development, including merge windows, release candidates, and final releases. Mainline kernels are designed to offer the latest features and improvements, making them attractive to developers and power users. Kernel.org lists the available Kernel versions.

Read More
Using Linux Kernel Memory Tiering

Using Linux Kernel Memory Tiering

In this post, I’ll discuss what memory tiering is, why we need it, and how to use the memory tiering feature available in the mainline v5.15 Kernel.

What is Memory Tiering?

With the advent of various new memory types, some systems will have multiple types of memory, e.g. High Bandwidth Memory (HBM), DRAM, Persistent Memory (PMem), CXL and others. The Memory Storage hierarchy should be familiar to you.

Memory Storage Hierarchy

Read More
How To Emulate CXL Devices using KVM and QEMU

How To Emulate CXL Devices using KVM and QEMU

What is CXL?

Compute Express Link (CXL) is an open standard for high-speed central processing unit-to-device and CPU-to-memory connections, designed for high-performance data center computers. CXL is built on the PCI Express physical and electrical interface with protocols in three areas: input/output, memory, and cache coherence.

CXL is designed to be an industry open standard interface for high-speed communications, as accelerators are increasingly used to complement CPUs in support of emerging applications such as Artificial Intelligence and Machine Learning.

Read More
How to Boot Linux from Intel® Optane™  Persistent Memory

How to Boot Linux from Intel® Optane™ Persistent Memory

Introduction

In this article, I will demonstrate how to configure a system with Intel Optane Persistent Memory (PMem) and use part of the PMem as a boot device. This little known feature can reduce boot times for those that need it.

The basic steps include:

  • Configure the Persistent Memory in AppDirect Interleaved
  • Create two small SECTOR namespaces, one per Region
  • Install the OS and select one or both of the namespaces (single disk install, or mirrored LVM)

Configure the Persistent Memory

The following figure shows how we will provision the persistent memory.

Read More
How to Boot Linux from Intel® Optane™  Persistent Memory

How to Boot Linux from Intel® Optane™ Persistent Memory

Introduction

In this article, I will demonstrate how to configure a system with Intel Optane Persistent Memory (PMem) and use part of the PMem as a boot device. This little known feature can reduce boot times for those that need it.

The basic steps include:

  • Configure the Persistent Memory in AppDirect Interleaved

  • Create two small SECTOR namespaces, one per Region

  • Install the OS and select one or both of the namespaces (single disk install, or mirrored LVM)

    Read More
How to build an upstream Fedora Kernel from source

How to build an upstream Fedora Kernel from source

I typically keep my Fedora system current, updating it once every week or two. More recently, I wanted to test the Idle Page Tracking feature, but this wasn’t enabled in the default kernel provided by Fedora.

# grep CONFIG_IDLE_PAGE_TRACKING /boot/config-$(uname -r)
# CONFIG_IDLE_PAGE_TRACKING is not set

To enable the feature, we need to build a custom kernel with the feature(s) we need. Thankfully, the process isn’t too difficult.

For this walk through, I’ll be building a customised version of the Fedora 32 kernel version I already have installed (5.8.7-200.fc32.x86_64), using some of the instructions from https://fedoraproject.org/wiki/Building_a_custom_kernel .

Read More
Linux Device Mapper WriteCache (dm-writecache) performance improvements in Linux Kernel 5.8

Linux Device Mapper WriteCache (dm-writecache) performance improvements in Linux Kernel 5.8

The Linux ‘dm-writecache’ target allows for writeback caching of newly written data to an SSD or NVMe using persistent memory will achieve much better performance in Linux Kernel 5.8.

Red Hat developer Mikulas Patocka has been working to enhance the dm-writecache performance using Intel Optane Persistent Memory (PMem) as the cache device.

The performance optimization now queued for Linux 5.8 is making use of CLFLUSHOPT within dm-writecache when available instead of MOVNTI. CLFLUSHOPT is one of Intel’s persistent memory instructions that allows for optimized flushing of cache lines by supporting greater concurrency. The CLFLUSHOPT instruction has been supported on Intel servers since Skylake and on AMD since Zen.

Read More

Intel Optane Persistent Memory Modules report "Non-functional" state in ipmctl

Issue

Executing ipmctl show-dimm to get device information shows the persistent memory modules in a ‘Non-functional’ health state, eg:

# ipmctl show -dimm

 DimmID | Capacity | HealthState    | ActionRequired | LockState | FWVersion
=============================================================================
 0x0001 | 0.0 GiB  | Non-functional | N/A            | N/A       | N/A
 0x0011 | 0.0 GiB  | Non-functional | N/A            | N/A       | N/A
 0x0021 | 0.0 GiB  | Non-functional | N/A            | N/A       | N/A
 0x0101 | 0.0 GiB  | Non-functional | N/A            | N/A       | N/A
 0x0111 | 0.0 GiB  | Non-functional | N/A            | N/A       | N/A
 0x0121 | 0.0 GiB  | Non-functional | N/A            | N/A       | N/A
 0x1001 | 0.0 GiB  | Non-functional | N/A            | N/A       | N/A
 0x1011 | 0.0 GiB  | Non-functional | N/A            | N/A       | N/A
 0x1021 | 0.0 GiB  | Non-functional | N/A            | N/A       | N/A
 0x1101 | 0.0 GiB  | Non-functional | N/A            | N/A       | N/A
 0x1111 | 0.0 GiB  | Non-functional | N/A            | N/A       | N/A
 0x1121 | 0.0 GiB  | Non-functional | N/A            | N/A       | N/A

Other ipmctl commands may fail and return “No functional DIMMs in the system.”, eg:

Read More
How To Verify Linux Kernel Support for Persistent Memory

How To Verify Linux Kernel Support for Persistent Memory

Linux Kernel support for persistent memory was first delivered in version 4.0 of the mainline kernel, however, it was not enabled by default until version 4.2.

If you use a Linux distribution that uses kernel 4.2 or later, or the distro backports features in to an older kernel, you will almost certainly have persistent memory support enabled by default. It is still worth verifying what features are enabled and disabled as this may vary by distro and release version for the very latest persistent memory features.

Read More
How To Extend Volatile System Memory (RAM) using Persistent Memory on Linux

How To Extend Volatile System Memory (RAM) using Persistent Memory on Linux

Intel(R) Optane(TM) Persistent Memory delivers a unique combination of affordable large capacity and support for data persistence. Electrically compatible with DDR4, large capacity modules up to 512GB each can be installed in compatible servers alongside DDR on the memory bus.

Intel’s persistent memory product can be provisioned in a volatile “Memory Mode” which replaces DRAM volatile capacity with the persistent memory capacity, and persistent “AppDirect” mode which presents both DRAM and persistent memory to the operating system and applications. Both modes are explained in more detail here . It is possible to configure a system to utilize a percentage of persistent memory as volatile and persistent, but this mixed-mode still provisions all the DRAM capacity as a Last-Level Cache.

Read More