<!DOCTYPE html>
<html lang="en" data-content_root="../../">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>GPU SVM Section — The Linux Kernel documentation</title>
<link rel="stylesheet" type="text/css" href="../../_static/pygments.css?v=fa44fd50" />
<link rel="stylesheet" type="text/css" href="../../_static/alabaster.css?v=3918102e" />
<script src="../../_static/documentation_options.js?v=5929fcd5"></script>
<script src="../../_static/doctools.js?v=9bcbadda"></script>
<script src="../../_static/sphinx_highlight.js?v=dc90522c"></script>
<link rel="index" title="Index" href="../../genindex.html" />
<link rel="search" title="Search" href="../../search.html" />
<link rel="next" title="I915 DG1/LMEM RFC Section" href="i915_gem_lmem.html" />
<link rel="prev" title="GPU RFC Section" href="index.html" />
<link rel="stylesheet" href="../../_static/custom.css" type="text/css" />
</head><body>
<div class="document">
<div class="sphinxsidebar" role="navigation" aria-label="Main">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../../index.html">
<img class="logo" src="../../_static/logo.svg" alt="Logo of The Linux Kernel"/>
</a></p>
<h1 class="logo"><a href="../../index.html">The Linux Kernel</a></h1>
<p class="blurb">6.18.50</p>
<search id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</search>
<script>document.getElementById('searchbox').style.display = "block"</script>
<p>
<h3 class="kernel-toc-contents">Contents</h3>
<input type="checkbox" class="kernel-toc-toggle" id = "kernel-toc-toggle" checked>
<label class="kernel-toc-title" for="kernel-toc-toggle"></label>
<div class="kerneltoc" id="kerneltoc">
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../process/development-process.html">Development process</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../process/submitting-patches.html">Submitting patches</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../process/code-of-conduct.html">Code of conduct</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../maintainer/index.html">Maintainer handbook</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../process/index.html">All development-process docs</a></li>
</ul>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../core-api/index.html">Core API</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../driver-api/index.html">Driver APIs</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../../subsystem-apis.html">Subsystems</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../../subsystem-apis.html#core-subsystems">Core subsystems</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="../../subsystem-apis.html#human-interfaces">Human interfaces</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="../../input/index.html">Input Documentation</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../hid/index.html">Human Interface Devices (HID)</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../sound/index.html">Sound Subsystem Documentation</a></li>
<li class="toctree-l3 current"><a class="reference internal" href="../index.html">GPU Driver Developer’s Guide</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../fb/index.html">Frame Buffer</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../leds/index.html">LEDs</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../subsystem-apis.html#networking-interfaces">Networking interfaces</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../subsystem-apis.html#storage-interfaces">Storage interfaces</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../subsystem-apis.html#other-subsystems">Other subsystems</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../locking/index.html">Locking</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../process/license-rules.html">Licensing rules</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../doc-guide/index.html">Writing documentation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev-tools/index.html">Development tools</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev-tools/testing-overview.html">Testing guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../kernel-hacking/index.html">Hacking guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../trace/index.html">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../fault-injection/index.html">Fault injection</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../livepatch/index.html">Livepatching</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rust/index.html">Rust</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../admin-guide/index.html">Administration</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../kbuild/index.html">Build system</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../admin-guide/reporting-issues.html">Reporting issues</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../tools/index.html">Userspace tools</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../userspace-api/index.html">Userspace API</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../firmware-guide/index.html">Firmware</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../devicetree/index.html">Firmware and Devicetree</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../arch/index.html">CPU architectures</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../staging/index.html">Unsorted documentation</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../translations/index.html">Translations</a></li>
</ul>
</div>
<script type="text/javascript"> <!--
var sbar = document.getElementsByClassName("sphinxsidebar")[0];
let currents = document.getElementsByClassName("current")
if (currents.length) {
sbar.scrollTop = currents[currents.length - 1].offsetTop;
}
--> </script>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../../_sources/gpu/rfc/gpusvm.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
</div>
</div>
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="gpu-svm-section">
<h1>GPU SVM Section<a class="headerlink" href="#gpu-svm-section" title="Link to this heading">¶</a></h1>
<section id="agreed-upon-design-principles">
<h2>Agreed upon design principles<a class="headerlink" href="#agreed-upon-design-principles" title="Link to this heading">¶</a></h2>
<ul class="simple">
<li><dl class="simple">
<dt>migrate_to_ram path</dt><dd><ul>
<li><p>Rely only on core MM concepts (migration PTEs, page references, and
page locking).</p></li>
<li><p>No driver specific locks other than locks for hardware interaction in
this path. These are not required and generally a bad idea to
invent driver defined locks to seal core MM races.</p></li>
<li><p>An example of a driver-specific lock causing issues occurred before
fixing do_swap_page to lock the faulting page. A driver-exclusive lock
in migrate_to_ram produced a stable livelock if enough threads read
the faulting page.</p></li>
<li><p>Partial migration is supported (i.e., a subset of pages attempting to
migrate can actually migrate, with only the faulting page guaranteed
to migrate).</p></li>
<li><p>Driver handles mixed migrations via retry loops rather than locking.</p></li>
</ul>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Eviction</dt><dd><ul>
<li><p>Eviction is defined as migrating data from the GPU back to the
CPU without a virtual address to free up GPU memory.</p></li>
<li><p>Only looking at physical memory data structures and locks as opposed to
looking at virtual memory data structures and locks.</p></li>
<li><p>No looking at mm/vma structs or relying on those being locked.</p></li>
<li><p>The rationale for the above two points is that CPU virtual addresses
can change at any moment, while the physical pages remain stable.</p></li>
<li><p>GPU page table invalidation, which requires a GPU virtual address, is
handled via the notifier that has access to the GPU virtual address.</p></li>
</ul>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>GPU fault side</dt><dd><ul>
<li><p>mmap_read only used around core MM functions which require this lock
and should strive to take mmap_read lock only in GPU SVM layer.</p></li>
<li><p>Big retry loop to handle all races with the mmu notifier under the gpu
pagetable locks/mmu notifier range lock/whatever we end up calling
those.</p></li>
<li><p>Races (especially against concurrent eviction or migrate_to_ram)
should not be handled on the fault side by trying to hold locks;
rather, they should be handled using retry loops. One possible
exception is holding a BO’s dma-resv lock during the initial migration
to VRAM, as this is a well-defined lock that can be taken underneath
the mmap_read lock.</p></li>
<li><p>One possible issue with the above approach is if a driver has a strict
migration policy requiring GPU access to occur in GPU memory.
Concurrent CPU access could cause a livelock due to endless retries.
While no current user (Xe) of GPU SVM has such a policy, it is likely
to be added in the future. Ideally, this should be resolved on the
core-MM side rather than through a driver-side lock.</p></li>
</ul>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Physical memory to virtual backpointer</dt><dd><ul>
<li><p>This does not work, as no pointers from physical memory to virtual
memory should exist. <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">mremap()</span></code> is an example of the core MM updating
the virtual address without notifying the driver of address
change rather the driver only receiving the invalidation notifier.</p></li>
<li><p>The physical memory backpointer (page->zone_device_data) should remain
stable from allocation to page free. Safely updating this against a
concurrent user would be very difficult unless the page is free.</p></li>
</ul>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>GPU pagetable locking</dt><dd><ul>
<li><p>Notifier lock only protects range tree, pages valid state for a range
(rather than seqno due to wider notifiers), pagetable entries, and
mmu notifier seqno tracking, it is not a global lock to protect
against races.</p></li>
<li><p>All races handled with big retry as mentioned above.</p></li>
</ul>
</dd>
</dl>
</li>
</ul>
</section>
<section id="overview-of-baseline-design">
<h2>Overview of baseline design<a class="headerlink" href="#overview-of-baseline-design" title="Link to this heading">¶</a></h2>
<p>GPU Shared Virtual Memory (GPU SVM) layer for the Direct Rendering Manager (DRM)
is a component of the DRM framework designed to manage shared virtual memory
between the CPU and GPU. It enables efficient data exchange and processing
for GPU-accelerated applications by allowing memory sharing and
synchronization between the CPU’s and GPU’s virtual address spaces.</p>
<p>Key GPU SVM Components:</p>
<ul class="simple">
<li><dl class="simple">
<dt>Notifiers:</dt><dd><p>Used for tracking memory intervals and notifying the GPU of changes,
notifiers are sized based on a GPU SVM initialization parameter, with a
recommendation of 512M or larger. They maintain a Red-BlacK tree and a
list of ranges that fall within the notifier interval. Notifiers are
tracked within a GPU SVM Red-BlacK tree and list and are dynamically
inserted or removed as ranges within the interval are created or
destroyed.</p>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Ranges:</dt><dd><p>Represent memory ranges mapped in a DRM device and managed by GPU SVM.
They are sized based on an array of chunk sizes, which is a GPU SVM
initialization parameter, and the CPU address space. Upon GPU fault,
the largest aligned chunk that fits within the faulting CPU address
space is chosen for the range size. Ranges are expected to be
dynamically allocated on GPU fault and removed on an MMU notifier UNMAP
event. As mentioned above, ranges are tracked in a notifier’s Red-Black
tree.</p>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Operations:</dt><dd><p>Define the interface for driver-specific GPU SVM operations such as
range allocation, notifier allocation, and invalidations.</p>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Device Memory Allocations:</dt><dd><p>Embedded structure containing enough information for GPU SVM to migrate
to / from device memory.</p>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Device Memory Operations:</dt><dd><p>Define the interface for driver-specific device memory operations
release memory, populate pfns, and copy to / from device memory.</p>
</dd>
</dl>
</li>
</ul>
<p>This layer provides interfaces for allocating, mapping, migrating, and
releasing memory ranges between the CPU and GPU. It handles all core memory
management interactions (DMA mapping, HMM, and migration) and provides
driver-specific virtual functions (vfuncs). This infrastructure is sufficient
to build the expected driver components for an SVM implementation as detailed
below.</p>
<p>Expected Driver Components:</p>
<ul class="simple">
<li><dl class="simple">
<dt>GPU page fault handler:</dt><dd><p>Used to create ranges and notifiers based on the fault address,
optionally migrate the range to device memory, and create GPU bindings.</p>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Garbage collector:</dt><dd><p>Used to unmap and destroy GPU bindings for ranges. Ranges are expected
to be added to the garbage collector upon a MMU_NOTIFY_UNMAP event in
notifier callback.</p>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Notifier callback:</dt><dd><p>Used to invalidate and DMA unmap GPU bindings for ranges.</p>
</dd>
</dl>
</li>
</ul>
<p>GPU SVM handles locking for core MM interactions, i.e., it locks/unlocks the
mmap lock as needed.</p>
<p>GPU SVM introduces a global notifier lock, which safeguards the notifier’s
range RB tree and list, as well as the range’s DMA mappings and sequence
number. GPU SVM manages all necessary locking and unlocking operations,
except for the recheck range’s pages being valid
(drm_gpusvm_range_pages_valid) when the driver is committing GPU bindings.
This lock corresponds to the <code class="docutils literal notranslate"><span class="pre">driver->update</span></code> lock mentioned in
<a class="reference internal" href="../../mm/hmm.html"><span class="doc">Heterogeneous Memory Management (HMM)</span></a>. Future revisions may transition from a GPU SVM
global lock to a per-notifier lock if finer-grained locking is deemed
necessary.</p>
<p>In addition to the locking mentioned above, the driver should implement a
lock to safeguard core GPU SVM function calls that modify state, such as
drm_gpusvm_range_find_or_insert and drm_gpusvm_range_remove. This lock is
denoted as ‘driver_svm_lock’ in code examples. Finer grained driver side
locking should also be possible for concurrent GPU fault processing within a
single GPU SVM. The ‘driver_svm_lock’ can be via drm_gpusvm_driver_set_lock
to add annotations to GPU SVM.</p>
<p>Partial unmapping of ranges (e.g., 1M out of 2M is unmapped by CPU resulting
in MMU_NOTIFY_UNMAP event) presents several challenges, with the main one
being that a subset of the range still has CPU and GPU mappings. If the
backing store for the range is in device memory, a subset of the backing
store has references. One option would be to split the range and device
memory backing store, but the implementation for this would be quite
complicated. Given that partial unmappings are rare and driver-defined range
sizes are relatively small, GPU SVM does not support splitting of ranges.</p>
<p>With no support for range splitting, upon partial unmapping of a range, the
driver is expected to invalidate and destroy the entire range. If the range
has device memory as its backing, the driver is also expected to migrate any
remaining pages back to RAM.</p>
<p>This section provides three examples of how to build the expected driver
components: the GPU page fault handler, the garbage collector, and the
notifier callback.</p>
<p>The generic code provided does not include logic for complex migration
policies, optimized invalidations, fined grained driver locking, or other
potentially required driver locking (e.g., DMA-resv locks).</p>
<ol class="arabic simple">
<li><p>GPU page fault handler</p></li>
</ol>
<div class="highlight-c notranslate"><div class="highlight"><pre><span></span><span class="kt">int</span><span class="w"> </span><span class="nf">driver_bind_range</span><span class="p">(</span><span class="k">struct</span><span class="w"> </span><span class="nc">drm_gpusvm</span><span class="w"> </span><span class="o">*</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">drm_gpusvm_range</span><span class="w"> </span><span class="o">*</span><span class="n">range</span><span class="p">)</span>
<span class="p">{</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">err</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span>
<span class="w"> </span><span class="n">driver_alloc_and_setup_memory_for_bind</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">range</span><span class="p">);</span>
<span class="w"> </span><span class="n">drm_gpusvm_notifier_lock</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">);</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">drm_gpusvm_range_pages_valid</span><span class="p">(</span><span class="n">range</span><span class="p">))</span>
<span class="w"> </span><span class="n">driver_commit_bind</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">range</span><span class="p">);</span>
<span class="w"> </span><span class="k">else</span>
<span class="w"> </span><span class="n">err</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">-</span><span class="n">EAGAIN</span><span class="p">;</span>
<span class="w"> </span><span class="n">drm_gpusvm_notifier_unlock</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">);</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">err</span><span class="p">;</span>
<span class="p">}</span>
<span class="kt">int</span><span class="w"> </span><span class="nf">driver_gpu_fault</span><span class="p">(</span><span class="k">struct</span><span class="w"> </span><span class="nc">drm_gpusvm</span><span class="w"> </span><span class="o">*</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="kt">unsigned</span><span class="w"> </span><span class="kt">long</span><span class="w"> </span><span class="n">fault_addr</span><span class="p">,</span>
<span class="w"> </span><span class="kt">unsigned</span><span class="w"> </span><span class="kt">long</span><span class="w"> </span><span class="n">gpuva_start</span><span class="p">,</span><span class="w"> </span><span class="kt">unsigned</span><span class="w"> </span><span class="kt">long</span><span class="w"> </span><span class="n">gpuva_end</span><span class="p">)</span>
<span class="p">{</span>
<span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">drm_gpusvm_ctx</span><span class="w"> </span><span class="n">ctx</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">{};</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="n">err</span><span class="p">;</span>
<span class="w"> </span><span class="n">driver_svm_lock</span><span class="p">();</span>
<span class="nl">retry</span><span class="p">:</span>
<span class="w"> </span><span class="c1">// Always process UNMAPs first so view of GPU SVM ranges is current</span>
<span class="w"> </span><span class="n">driver_garbage_collector</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">);</span>
<span class="w"> </span><span class="n">range</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">drm_gpusvm_range_find_or_insert</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">fault_addr</span><span class="p">,</span>
<span class="w"> </span><span class="n">gpuva_start</span><span class="p">,</span><span class="w"> </span><span class="n">gpuva_end</span><span class="p">,</span>
<span class="w"> </span><span class="o">&</span><span class="n">ctx</span><span class="p">);</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">IS_ERR</span><span class="p">(</span><span class="n">range</span><span class="p">))</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">err</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">PTR_ERR</span><span class="p">(</span><span class="n">range</span><span class="p">);</span>
<span class="w"> </span><span class="k">goto</span><span class="w"> </span><span class="n">unlock</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">driver_migration_policy</span><span class="p">(</span><span class="n">range</span><span class="p">))</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">err</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">drm_pagemap_populate_mm</span><span class="p">(</span><span class="n">driver_choose_drm_pagemap</span><span class="p">(),</span>
<span class="w"> </span><span class="n">gpuva_start</span><span class="p">,</span><span class="w"> </span><span class="n">gpuva_end</span><span class="p">,</span><span class="w"> </span><span class="n">gpusvm</span><span class="o">-></span><span class="n">mm</span><span class="p">,</span>
<span class="w"> </span><span class="n">ctx</span><span class="o">-></span><span class="n">timeslice_ms</span><span class="p">);</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">err</span><span class="p">)</span><span class="w"> </span><span class="c1">// CPU mappings may have changed</span>
<span class="w"> </span><span class="k">goto</span><span class="w"> </span><span class="n">retry</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">err</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">drm_gpusvm_range_get_pages</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">range</span><span class="p">,</span><span class="w"> </span><span class="o">&</span><span class="n">ctx</span><span class="p">);</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">err</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="o">-</span><span class="n">EOPNOTSUPP</span><span class="w"> </span><span class="o">||</span><span class="w"> </span><span class="n">err</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="o">-</span><span class="n">EFAULT</span><span class="w"> </span><span class="o">||</span><span class="w"> </span><span class="n">err</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="o">-</span><span class="n">EPERM</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="c1">// CPU mappings changed</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">err</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="o">-</span><span class="n">EOPNOTSUPP</span><span class="p">)</span>
<span class="w"> </span><span class="n">drm_gpusvm_range_evict</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">range</span><span class="p">);</span>
<span class="w"> </span><span class="k">goto</span><span class="w"> </span><span class="n">retry</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">err</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">goto</span><span class="w"> </span><span class="n">unlock</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="n">err</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">driver_bind_range</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">range</span><span class="p">);</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">err</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="o">-</span><span class="n">EAGAIN</span><span class="p">)</span><span class="w"> </span><span class="c1">// CPU mappings changed</span>
<span class="w"> </span><span class="k">goto</span><span class="w"> </span><span class="n">retry</span>
<span class="nl">unlock</span><span class="p">:</span>
<span class="w"> </span><span class="n">driver_svm_unlock</span><span class="p">();</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">err</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<ol class="arabic simple" start="2">
<li><p>Garbage Collector</p></li>
</ol>
<div class="highlight-c notranslate"><div class="highlight"><pre><span></span><span class="kt">void</span><span class="w"> </span><span class="nf">__driver_garbage_collector</span><span class="p">(</span><span class="k">struct</span><span class="w"> </span><span class="nc">drm_gpusvm</span><span class="w"> </span><span class="o">*</span><span class="n">gpusvm</span><span class="p">,</span>
<span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">drm_gpusvm_range</span><span class="w"> </span><span class="o">*</span><span class="n">range</span><span class="p">)</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">assert_driver_svm_locked</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">);</span>
<span class="w"> </span><span class="c1">// Partial unmap, migrate any remaining device memory pages back to RAM</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">range</span><span class="o">-></span><span class="n">flags</span><span class="p">.</span><span class="n">partial_unmap</span><span class="p">)</span>
<span class="w"> </span><span class="n">drm_gpusvm_range_evict</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">range</span><span class="p">);</span>
<span class="w"> </span><span class="n">driver_unbind_range</span><span class="p">(</span><span class="n">range</span><span class="p">);</span>
<span class="w"> </span><span class="n">drm_gpusvm_range_remove</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">range</span><span class="p">);</span>
<span class="p">}</span>
<span class="kt">void</span><span class="w"> </span><span class="nf">driver_garbage_collector</span><span class="p">(</span><span class="k">struct</span><span class="w"> </span><span class="nc">drm_gpusvm</span><span class="w"> </span><span class="o">*</span><span class="n">gpusvm</span><span class="p">)</span>
<span class="p">{</span>
<span class="w"> </span><span class="n">assert_driver_svm_locked</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">);</span>
<span class="w"> </span><span class="n">for_each_range_in_garbage_collector</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">range</span><span class="p">)</span>
<span class="w"> </span><span class="n">__driver_garbage_collector</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">range</span><span class="p">);</span>
<span class="p">}</span>
</pre></div>
</div>
<ol class="arabic simple" start="3">
<li><p>Notifier callback</p></li>
</ol>
<div class="highlight-c notranslate"><div class="highlight"><pre><span></span><span class="kt">void</span><span class="w"> </span><span class="nf">driver_invalidation</span><span class="p">(</span><span class="k">struct</span><span class="w"> </span><span class="nc">drm_gpusvm</span><span class="w"> </span><span class="o">*</span><span class="n">gpusvm</span><span class="p">,</span>
<span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">drm_gpusvm_notifier</span><span class="w"> </span><span class="o">*</span><span class="n">notifier</span><span class="p">,</span>
<span class="w"> </span><span class="k">const</span><span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">mmu_notifier_range</span><span class="w"> </span><span class="o">*</span><span class="n">mmu_range</span><span class="p">)</span>
<span class="p">{</span>
<span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">drm_gpusvm_ctx</span><span class="w"> </span><span class="n">ctx</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="p">.</span><span class="n">in_notifier</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">true</span><span class="p">,</span><span class="w"> </span><span class="p">};</span>
<span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">drm_gpusvm_range</span><span class="w"> </span><span class="o">*</span><span class="n">range</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">NULL</span><span class="p">;</span>
<span class="w"> </span><span class="n">driver_invalidate_device_pages</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">mmu_range</span><span class="o">-></span><span class="n">start</span><span class="p">,</span><span class="w"> </span><span class="n">mmu_range</span><span class="o">-></span><span class="n">end</span><span class="p">);</span>
<span class="w"> </span><span class="n">drm_gpusvm_for_each_range</span><span class="p">(</span><span class="n">range</span><span class="p">,</span><span class="w"> </span><span class="n">notifier</span><span class="p">,</span><span class="w"> </span><span class="n">mmu_range</span><span class="o">-></span><span class="n">start</span><span class="p">,</span>
<span class="w"> </span><span class="n">mmu_range</span><span class="o">-></span><span class="n">end</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">drm_gpusvm_range_unmap_pages</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">range</span><span class="p">,</span><span class="w"> </span><span class="o">&</span><span class="n">ctx</span><span class="p">);</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">mmu_range</span><span class="o">-></span><span class="n">event</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="n">MMU_NOTIFY_UNMAP</span><span class="p">)</span>
<span class="w"> </span><span class="k">continue</span><span class="p">;</span>
<span class="w"> </span><span class="n">drm_gpusvm_range_set_unmapped</span><span class="p">(</span><span class="n">range</span><span class="p">,</span><span class="w"> </span><span class="n">mmu_range</span><span class="p">);</span>
<span class="w"> </span><span class="n">driver_garbage_collector_add</span><span class="p">(</span><span class="n">gpusvm</span><span class="p">,</span><span class="w"> </span><span class="n">range</span><span class="p">);</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
</section>
<section id="overview-of-drm-pagemap-design">
<h2>Overview of drm_pagemap design<a class="headerlink" href="#overview-of-drm-pagemap-design" title="Link to this heading">¶</a></h2>
<p>The DRM pagemap layer is intended to augment the dev_pagemap functionality by
providing a way to populate a <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">mm_struct</span></code> virtual range with device
private pages and to provide helpers to abstract device memory allocations,
to migrate memory back and forth between device memory and system RAM and
to handle access (and in the future migration) between devices implementing
a fast interconnect that is not necessarily visible to the rest of the
system.</p>
<p>Typically the DRM pagemap receives requests from one or more DRM GPU SVM
instances to populate <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">mm_struct</span></code> virtual ranges with memory, and the
migration is best effort only and may thus fail. The implementation should
also handle device unbinding by blocking (return an -ENODEV) error for new
population requests and after that migrate all device pages to system ram.</p>
<p>Migration granularity typically follows the GPU SVM range requests, but
if there are clashes, due to races or due to the fact that multiple GPU
SVM instances have different views of the ranges used, and because of that
parts of a requested range is already present in the requested device memory,
the implementation has a variety of options. It can fail and it can choose
to populate only the part of the range that isn’t already in device memory,
and it can evict the range to system before trying to migrate. Ideally an
implementation would just try to migrate the missing part of the range and
allocate just enough memory to do so.</p>
<p>When migrating to system memory as a response to a cpu fault or a device
memory eviction request, currently a full device memory allocation is
migrated back to system. Moving forward this might need improvement for
situations where a single page needs bouncing between system memory and
device memory due to, for example, atomic operations.</p>
<p>Key DRM pagemap components:</p>
<ul class="simple">
<li><dl class="simple">
<dt>Device Memory Allocations:</dt><dd><p>Embedded structure containing enough information for the drm_pagemap to
migrate to / from device memory.</p>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Device Memory Operations:</dt><dd><p>Define the interface for driver-specific device memory operations
release memory, populate pfns, and copy to / from device memory.</p>
</dd>
</dl>
</li>
</ul>
</section>
<section id="possible-future-design-features">
<h2>Possible future design features<a class="headerlink" href="#possible-future-design-features" title="Link to this heading">¶</a></h2>
<ul class="simple">
<li><dl class="simple">
<dt>Concurrent GPU faults</dt><dd><ul>
<li><p>CPU faults are concurrent so makes sense to have concurrent GPU
faults.</p></li>
<li><p>Should be possible with fined grained locking in the driver GPU
fault handler.</p></li>
<li><p>No expected GPU SVM changes required.</p></li>
</ul>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Ranges with mixed system and device pages</dt><dd><ul>
<li><p>Can be added if required to drm_gpusvm_get_pages fairly easily.</p></li>
</ul>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Multi-GPU support</dt><dd><ul>
<li><p>Work in progress and patches expected after initially landing on GPU
SVM.</p></li>
<li><p>Ideally can be done with little to no changes to GPU SVM.</p></li>
</ul>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Drop ranges in favor of radix tree</dt><dd><ul>
<li><p>May be desirable for faster notifiers.</p></li>
</ul>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Compound device pages</dt><dd><ul>
<li><p>Nvidia, AMD, and Intel all have agreed expensive core MM functions in
migrate device layer are a performance bottleneck, having compound
device pages should help increase performance by reducing the number
of these expensive calls.</p></li>
</ul>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Higher order dma mapping for migration</dt><dd><ul>
<li><p>4k dma mapping adversely affects migration performance on Intel
hardware, higher order (2M) dma mapping should help here.</p></li>
</ul>
</dd>
</dl>
</li>
<li><p>Build common userptr implementation on top of GPU SVM</p></li>
<li><p>Driver side madvise implementation and migration policies</p></li>
<li><p>Pull in pending dma-mapping API changes from Leon / Nvidia when these land</p></li>
</ul>
</section>
</section>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©The kernel development community.
|
Powered by <a href="https://www.sphinx-doc.org/">Sphinx 8.1.3</a>
& <a href="https://alabaster.readthedocs.io">Alabaster 0.7.16</a>
|
<a href="../../_sources/gpu/rfc/gpusvm.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>