<!DOCTYPE html>
<html lang="en" data-content_root="../../">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>1. Library Design — The Linux Kernel documentation</title>
<link rel="stylesheet" type="text/css" href="../../_static/pygments.css?v=fa44fd50" />
<link rel="stylesheet" type="text/css" href="../../_static/alabaster.css?v=3918102e" />
<script src="../../_static/documentation_options.js?v=5929fcd5"></script>
<script src="../../_static/doctools.js?v=9bcbadda"></script>
<script src="../../_static/sphinx_highlight.js?v=dc90522c"></script>
<link rel="index" title="Index" href="../../genindex.html" />
<link rel="search" title="Search" href="../../search.html" />
<link rel="next" title="2. Supported File Operations" href="operations.html" />
<link rel="prev" title="VFS iomap Documentation" href="index.html" />
<link rel="stylesheet" href="../../_static/custom.css" type="text/css" />
</head><body>
<div class="document">
<div class="sphinxsidebar" role="navigation" aria-label="Main">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../../index.html">
<img class="logo" src="../../_static/logo.svg" alt="Logo of The Linux Kernel"/>
</a></p>
<h1 class="logo"><a href="../../index.html">The Linux Kernel</a></h1>
<p class="blurb">6.18.50</p>
<search id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</search>
<script>document.getElementById('searchbox').style.display = "block"</script>
<p>
<h3 class="kernel-toc-contents">Contents</h3>
<input type="checkbox" class="kernel-toc-toggle" id = "kernel-toc-toggle" checked>
<label class="kernel-toc-title" for="kernel-toc-toggle"></label>
<div class="kerneltoc" id="kerneltoc">
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../process/development-process.html">Development process</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../process/submitting-patches.html">Submitting patches</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../process/code-of-conduct.html">Code of conduct</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../maintainer/index.html">Maintainer handbook</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../process/index.html">All development-process docs</a></li>
</ul>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../core-api/index.html">Core API</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../driver-api/index.html">Driver APIs</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../../subsystem-apis.html">Subsystems</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../../subsystem-apis.html#core-subsystems">Core subsystems</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../subsystem-apis.html#human-interfaces">Human interfaces</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../subsystem-apis.html#networking-interfaces">Networking interfaces</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="../../subsystem-apis.html#storage-interfaces">Storage interfaces</a><ul class="current">
<li class="toctree-l3 current"><a class="reference internal" href="../index.html">Filesystems in the Linux kernel</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../block/index.html">Block</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../cdrom/index.html">CD-ROM</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../scsi/index.html">SCSI Subsystem</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../target/index.html">TCM Virtual Device</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../nvme/index.html">NVMe Subsystem</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../subsystem-apis.html#other-subsystems">Other subsystems</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../locking/index.html">Locking</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../process/license-rules.html">Licensing rules</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../doc-guide/index.html">Writing documentation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev-tools/index.html">Development tools</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev-tools/testing-overview.html">Testing guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../kernel-hacking/index.html">Hacking guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../trace/index.html">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../fault-injection/index.html">Fault injection</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../livepatch/index.html">Livepatching</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rust/index.html">Rust</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../admin-guide/index.html">Administration</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../kbuild/index.html">Build system</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../admin-guide/reporting-issues.html">Reporting issues</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../tools/index.html">Userspace tools</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../userspace-api/index.html">Userspace API</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../firmware-guide/index.html">Firmware</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../devicetree/index.html">Firmware and Devicetree</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../arch/index.html">CPU architectures</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../staging/index.html">Unsorted documentation</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../translations/index.html">Translations</a></li>
</ul>
</div>
<script type="text/javascript"> <!--
var sbar = document.getElementsByClassName("sphinxsidebar")[0];
let currents = document.getElementsByClassName("current")
if (currents.length) {
sbar.scrollTop = currents[currents.length - 1].offsetTop;
}
--> </script>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../../_sources/filesystems/iomap/design.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
</div>
</div>
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<span class="target" id="iomap-design"></span><section id="library-design">
<h1><span class="section-number">1. </span>Library Design<a class="headerlink" href="#library-design" title="Link to this heading">¶</a></h1>
<nav class="contents local" id="table-of-contents">
<p class="topic-title">Table of Contents</p>
<ul class="simple">
<li><p><a class="reference internal" href="#introduction" id="id1">Introduction</a></p></li>
<li><p><a class="reference internal" href="#who-should-read-this" id="id2">Who Should Read This?</a></p></li>
<li><p><a class="reference internal" href="#how-is-this-better" id="id3">How Is This Better?</a></p></li>
<li><p><a class="reference internal" href="#file-range-iterator" id="id4">File Range Iterator</a></p>
<ul>
<li><p><a class="reference internal" href="#definitions" id="id5">Definitions</a></p></li>
<li><p><a class="reference internal" href="#struct-iomap" id="id6"><code class="docutils literal notranslate"><span class="pre">struct</span> <span class="pre">iomap</span></code></a></p></li>
<li><p><a class="reference internal" href="#struct-iomap-ops" id="id7"><code class="docutils literal notranslate"><span class="pre">struct</span> <span class="pre">iomap_ops</span></code></a></p>
<ul>
<li><p><a class="reference internal" href="#iomap-begin" id="id8"><code class="docutils literal notranslate"><span class="pre">->iomap_begin</span></code></a></p></li>
<li><p><a class="reference internal" href="#iomap-end" id="id9"><code class="docutils literal notranslate"><span class="pre">->iomap_end</span></code></a></p></li>
</ul>
</li>
</ul>
</li>
<li><p><a class="reference internal" href="#preparing-for-file-operations" id="id10">Preparing for File Operations</a></p></li>
<li><p><a class="reference internal" href="#locking-hierarchy" id="id11">Locking Hierarchy</a></p></li>
<li><p><a class="reference internal" href="#bugs-and-limitations" id="id12">Bugs and Limitations</a></p></li>
</ul>
</nav>
<section id="introduction">
<h2><a class="toc-backref" href="#id1" role="doc-backlink"><span class="section-number">1.1. </span>Introduction</a><a class="headerlink" href="#introduction" title="Link to this heading">¶</a></h2>
<p>iomap is a filesystem library for handling common file operations.
The library has two layers:</p>
<blockquote>
<div><ol class="arabic simple">
<li><p>A lower layer that provides an iterator over ranges of file offsets.
This layer tries to obtain mappings of each file ranges to storage
from the filesystem, but the storage information is not necessarily
required.</p></li>
<li><p>An upper layer that acts upon the space mappings provided by the
lower layer iterator.</p></li>
</ol>
</div></blockquote>
<p>The iteration can involve mappings of file’s logical offset ranges to
physical extents, but the storage layer information is not necessarily
required, e.g. for walking cached file information.
The library exports various APIs for implementing file operations such
as:</p>
<blockquote>
<div><ul class="simple">
<li><p>Pagecache reads and writes</p></li>
<li><p>Folio write faults to the pagecache</p></li>
<li><p>Writeback of dirty folios</p></li>
<li><p>Direct I/O reads and writes</p></li>
<li><p>fsdax I/O reads, writes, loads, and stores</p></li>
<li><p>FIEMAP</p></li>
<li><p>lseek <code class="docutils literal notranslate"><span class="pre">SEEK_DATA</span></code> and <code class="docutils literal notranslate"><span class="pre">SEEK_HOLE</span></code></p></li>
<li><p>swapfile activation</p></li>
</ul>
</div></blockquote>
<p>This origins of this library is the file I/O path that XFS once used; it
has now been extended to cover several other operations.</p>
</section>
<section id="who-should-read-this">
<h2><a class="toc-backref" href="#id2" role="doc-backlink"><span class="section-number">1.2. </span>Who Should Read This?</a><a class="headerlink" href="#who-should-read-this" title="Link to this heading">¶</a></h2>
<p>The target audience for this document are filesystem, storage, and
pagecache programmers and code reviewers.</p>
<p>If you are working on PCI, machine architectures, or device drivers, you
are most likely in the wrong place.</p>
</section>
<section id="how-is-this-better">
<h2><a class="toc-backref" href="#id3" role="doc-backlink"><span class="section-number">1.3. </span>How Is This Better?</a><a class="headerlink" href="#how-is-this-better" title="Link to this heading">¶</a></h2>
<p>Unlike the classic Linux I/O model which breaks file I/O into small
units (generally memory pages or blocks) and looks up space mappings on
the basis of that unit, the iomap model asks the filesystem for the
largest space mappings that it can create for a given file operation and
initiates operations on that basis.
This strategy improves the filesystem’s visibility into the size of the
operation being performed, which enables it to combat fragmentation with
larger space allocations when possible.
Larger space mappings improve runtime performance by amortizing the cost
of mapping function calls into the filesystem across a larger amount of
data.</p>
<p>At a high level, an iomap operation <a class="reference external" href="https://lore.kernel.org/all/ZGbVaewzcCysclPt@dread.disaster.area/">looks like this</a>:</p>
<ol class="arabic simple">
<li><p>For each byte in the operation range...</p>
<ol class="arabic simple">
<li><p>Obtain a space mapping via <code class="docutils literal notranslate"><span class="pre">->iomap_begin</span></code></p></li>
<li><p>For each sub-unit of work...</p>
<ol class="arabic simple">
<li><p>Revalidate the mapping and go back to (1) above, if necessary.
So far only the pagecache operations need to do this.</p></li>
<li><p>Do the work</p></li>
</ol>
</li>
<li><p>Increment operation cursor</p></li>
<li><p>Release the mapping via <code class="docutils literal notranslate"><span class="pre">->iomap_end</span></code>, if necessary</p></li>
</ol>
</li>
</ol>
<p>Each iomap operation will be covered in more detail below.
This library was covered previously by an <a class="reference external" href="https://lwn.net/Articles/935934/">LWN article</a> and a <a class="reference external" href="https://kernelnewbies.org/KernelProjects/iomap">KernelNewbies page</a>.</p>
<p>The goal of this document is to provide a brief discussion of the
design and capabilities of iomap, followed by a more detailed catalog
of the interfaces presented by iomap.
If you change iomap, please update this design document.</p>
</section>
<section id="file-range-iterator">
<h2><a class="toc-backref" href="#id4" role="doc-backlink"><span class="section-number">1.4. </span>File Range Iterator</a><a class="headerlink" href="#file-range-iterator" title="Link to this heading">¶</a></h2>
<section id="definitions">
<h3><a class="toc-backref" href="#id5" role="doc-backlink"><span class="section-number">1.4.1. </span>Definitions</a><a class="headerlink" href="#definitions" title="Link to this heading">¶</a></h3>
<blockquote>
<div><ul class="simple">
<li><p><strong>buffer head</strong>: Shattered remnants of the old buffer cache.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">fsblock</span></code>: The block size of a file, also known as <code class="docutils literal notranslate"><span class="pre">i_blocksize</span></code>.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">i_rwsem</span></code>: The VFS <code class="docutils literal notranslate"><span class="pre">struct</span> <span class="pre">inode</span></code> rwsemaphore.
Processes hold this in shared mode to read file state and contents.
Some filesystems may allow shared mode for writes.
Processes often hold this in exclusive mode to change file state and
contents.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">invalidate_lock</span></code>: The pagecache <code class="docutils literal notranslate"><span class="pre">struct</span> <span class="pre">address_space</span></code>
rwsemaphore that protects against folio insertion and removal for
filesystems that support punching out folios below EOF.
Processes wishing to insert folios must hold this lock in shared
mode to prevent removal, though concurrent insertion is allowed.
Processes wishing to remove folios must hold this lock in exclusive
mode to prevent insertions.
Concurrent removals are not allowed.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">dax_read_lock</span></code>: The RCU read lock that dax takes to prevent a
device pre-shutdown hook from returning before other threads have
released resources.</p></li>
<li><p><strong>filesystem mapping lock</strong>: This synchronization primitive is
internal to the filesystem and must protect the file mapping data
from updates while a mapping is being sampled.
The filesystem author must determine how this coordination should
happen; it does not need to be an actual lock.</p></li>
<li><p><strong>iomap internal operation lock</strong>: This is a general term for
synchronization primitives that iomap functions take while holding a
mapping.
A specific example would be taking the folio lock while reading or
writing the pagecache.</p></li>
<li><p><strong>pure overwrite</strong>: A write operation that does not require any
metadata or zeroing operations to perform during either submission
or completion.
This implies that the filesystem must have already allocated space
on disk as <code class="docutils literal notranslate"><span class="pre">IOMAP_MAPPED</span></code> and the filesystem must not place any
constraints on IO alignment or size.
The only constraints on I/O alignment are device level (minimum I/O
size and alignment, typically sector size).</p></li>
</ul>
</div></blockquote>
</section>
<section id="struct-iomap">
<h3><a class="toc-backref" href="#id6" role="doc-backlink"><span class="section-number">1.4.2. </span><code class="docutils literal notranslate"><span class="pre">struct</span> <span class="pre">iomap</span></code></a><a class="headerlink" href="#struct-iomap" title="Link to this heading">¶</a></h3>
<p>The filesystem communicates to the iomap iterator the mapping of
byte ranges of a file to byte ranges of a storage device with the
structure below:</p>
<div class="highlight-c notranslate"><div class="highlight"><pre><span></span><span class="k">struct</span><span class="w"> </span><span class="nc">iomap</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">u64</span><span class="w"> </span><span class="n">addr</span><span class="p">;</span>
<span class="w"> </span><span class="n">loff_t</span><span class="w"> </span><span class="n">offset</span><span class="p">;</span>
<span class="w"> </span><span class="n">u64</span><span class="w"> </span><span class="n">length</span><span class="p">;</span>
<span class="w"> </span><span class="n">u16</span><span class="w"> </span><span class="n">type</span><span class="p">;</span>
<span class="w"> </span><span class="n">u16</span><span class="w"> </span><span class="n">flags</span><span class="p">;</span>
<span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">block_device</span><span class="w"> </span><span class="o">*</span><span class="n">bdev</span><span class="p">;</span>
<span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">dax_device</span><span class="w"> </span><span class="o">*</span><span class="n">dax_dev</span><span class="p">;</span>
<span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="o">*</span><span class="n">inline_data</span><span class="p">;</span>
<span class="w"> </span><span class="kt">void</span><span class="w"> </span><span class="o">*</span><span class="n">private</span><span class="p">;</span>
<span class="w"> </span><span class="n">u64</span><span class="w"> </span><span class="n">validity_cookie</span><span class="p">;</span>
<span class="p">};</span>
</pre></div>
</div>
<p>The fields are as follows:</p>
<blockquote>
<div><ul>
<li><p><code class="docutils literal notranslate"><span class="pre">offset</span></code> and <code class="docutils literal notranslate"><span class="pre">length</span></code> describe the range of file offsets, in
bytes, covered by this mapping.
These fields must always be set by the filesystem.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">type</span></code> describes the type of the space mapping:</p>
<ul class="simple">
<li><p><strong>IOMAP_HOLE</strong>: No storage has been allocated.
This type must never be returned in response to an <code class="docutils literal notranslate"><span class="pre">IOMAP_WRITE</span></code>
operation because writes must allocate and map space, and return
the mapping.
The <code class="docutils literal notranslate"><span class="pre">addr</span></code> field must be set to <code class="docutils literal notranslate"><span class="pre">IOMAP_NULL_ADDR</span></code>.
iomap does not support writing (whether via pagecache or direct
I/O) to a hole.</p></li>
<li><p><strong>IOMAP_DELALLOC</strong>: A promise to allocate space at a later time
(“delayed allocation”).
If the filesystem returns IOMAP_F_NEW here and the write fails, the
<code class="docutils literal notranslate"><span class="pre">->iomap_end</span></code> function must delete the reservation.
The <code class="docutils literal notranslate"><span class="pre">addr</span></code> field must be set to <code class="docutils literal notranslate"><span class="pre">IOMAP_NULL_ADDR</span></code>.</p></li>
<li><p><strong>IOMAP_MAPPED</strong>: The file range maps to specific space on the
storage device.
The device is returned in <code class="docutils literal notranslate"><span class="pre">bdev</span></code> or <code class="docutils literal notranslate"><span class="pre">dax_dev</span></code>.
The device address, in bytes, is returned via <code class="docutils literal notranslate"><span class="pre">addr</span></code>.</p></li>
<li><p><strong>IOMAP_UNWRITTEN</strong>: The file range maps to specific space on the
storage device, but the space has not yet been initialized.
The device is returned in <code class="docutils literal notranslate"><span class="pre">bdev</span></code> or <code class="docutils literal notranslate"><span class="pre">dax_dev</span></code>.
The device address, in bytes, is returned via <code class="docutils literal notranslate"><span class="pre">addr</span></code>.
Reads from this type of mapping will return zeroes to the caller.
For a write or writeback operation, the ioend should update the
mapping to MAPPED.
Refer to the sections about ioends for more details.</p></li>
<li><p><strong>IOMAP_INLINE</strong>: The file range maps to the memory buffer
specified by <code class="docutils literal notranslate"><span class="pre">inline_data</span></code>.
For write operation, the <code class="docutils literal notranslate"><span class="pre">->iomap_end</span></code> function presumably
handles persisting the data.
The <code class="docutils literal notranslate"><span class="pre">addr</span></code> field must be set to <code class="docutils literal notranslate"><span class="pre">IOMAP_NULL_ADDR</span></code>.</p></li>
</ul>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">flags</span></code> describe the status of the space mapping.
These flags should be set by the filesystem in <code class="docutils literal notranslate"><span class="pre">->iomap_begin</span></code>:</p>
<ul class="simple">
<li><p><strong>IOMAP_F_NEW</strong>: The space under the mapping is newly allocated.
Areas that will not be written to must be zeroed.
If a write fails and the mapping is a space reservation, the
reservation must be deleted.</p></li>
<li><p><strong>IOMAP_F_DIRTY</strong>: The inode will have uncommitted metadata needed
to access any data written.
fdatasync is required to commit these changes to persistent
storage.
This needs to take into account metadata changes that <em>may</em> be made
at I/O completion, such as file size updates from direct I/O.</p></li>
<li><p><strong>IOMAP_F_SHARED</strong>: The space under the mapping is shared.
Copy on write is necessary to avoid corrupting other file data.</p></li>
<li><p><strong>IOMAP_F_BUFFER_HEAD</strong>: This mapping requires the use of buffer
heads for pagecache operations.
Do not add more uses of this.</p></li>
<li><p><strong>IOMAP_F_MERGED</strong>: Multiple contiguous block mappings were
coalesced into this single mapping.
This is only useful for FIEMAP.</p></li>
<li><p><strong>IOMAP_F_XATTR</strong>: The mapping is for extended attribute data, not
regular file data.
This is only useful for FIEMAP.</p></li>
<li><p><strong>IOMAP_F_BOUNDARY</strong>: This indicates I/O and its completion must not be
merged with any other I/O or completion. Filesystems must use this when
submitting I/O to devices that cannot handle I/O crossing certain LBAs
(e.g. ZNS devices). This flag applies only to buffered I/O writeback; all
other functions ignore it.</p></li>
<li><p><strong>IOMAP_F_PRIVATE</strong>: This flag is reserved for filesystem private use.</p></li>
<li><p><strong>IOMAP_F_ANON_WRITE</strong>: Indicates that (write) I/O does not have a target
block assigned to it yet and the file system will do that in the bio
submission handler, splitting the I/O as needed.</p></li>
<li><p><strong>IOMAP_F_ATOMIC_BIO</strong>: This indicates write I/O must be submitted with the
<code class="docutils literal notranslate"><span class="pre">REQ_ATOMIC</span></code> flag set in the bio. Filesystems need to set this flag to
inform iomap that the write I/O operation requires torn-write protection
based on HW-offload mechanism. They must also ensure that mapping updates
upon the completion of the I/O must be performed in a single metadata
update.</p></li>
</ul>
<p>These flags can be set by iomap itself during file operations.
The filesystem should supply an <code class="docutils literal notranslate"><span class="pre">->iomap_end</span></code> function if it needs
to observe these flags:</p>
<ul class="simple">
<li><p><strong>IOMAP_F_SIZE_CHANGED</strong>: The file size has changed as a result of
using this mapping.</p></li>
<li><p><strong>IOMAP_F_STALE</strong>: The mapping was found to be stale.
iomap will call <code class="docutils literal notranslate"><span class="pre">->iomap_end</span></code> on this mapping and then
<code class="docutils literal notranslate"><span class="pre">->iomap_begin</span></code> to obtain a new mapping.</p></li>
</ul>
<p>Currently, these flags are only set by pagecache operations.</p>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">addr</span></code> describes the device address, in bytes.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">bdev</span></code> describes the block device for this mapping.
This only needs to be set for mapped or unwritten operations.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">dax_dev</span></code> describes the DAX device for this mapping.
This only needs to be set for mapped or unwritten operations, and
only for a fsdax operation.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">inline_data</span></code> points to a memory buffer for I/O involving
<code class="docutils literal notranslate"><span class="pre">IOMAP_INLINE</span></code> mappings.
This value is ignored for all other mapping types.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">private</span></code> is a pointer to <a class="reference external" href="https://lore.kernel.org/all/20180619164137.13720-7-hch@lst.de/">filesystem-private information</a>.
This value will be passed unchanged to <code class="docutils literal notranslate"><span class="pre">->iomap_end</span></code>.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">validity_cookie</span></code> is a magic freshness value set by the filesystem
that should be used to detect stale mappings.
For pagecache operations this is critical for correct operation
because page faults can occur, which implies that filesystem locks
should not be held between <code class="docutils literal notranslate"><span class="pre">->iomap_begin</span></code> and <code class="docutils literal notranslate"><span class="pre">->iomap_end</span></code>.
Filesystems with completely static mappings need not set this value.
Only pagecache operations revalidate mappings; see the section about
<code class="docutils literal notranslate"><span class="pre">iomap_valid</span></code> for details.</p></li>
</ul>
</div></blockquote>
</section>
<section id="struct-iomap-ops">
<h3><a class="toc-backref" href="#id7" role="doc-backlink"><span class="section-number">1.4.3. </span><code class="docutils literal notranslate"><span class="pre">struct</span> <span class="pre">iomap_ops</span></code></a><a class="headerlink" href="#struct-iomap-ops" title="Link to this heading">¶</a></h3>
<p>Every iomap function requires the filesystem to pass an operations
structure to obtain a mapping and (optionally) to release the mapping:</p>
<div class="highlight-c notranslate"><div class="highlight"><pre><span></span><span class="k">struct</span><span class="w"> </span><span class="nc">iomap_ops</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="p">(</span><span class="o">*</span><span class="n">iomap_begin</span><span class="p">)(</span><span class="k">struct</span><span class="w"> </span><span class="nc">inode</span><span class="w"> </span><span class="o">*</span><span class="n">inode</span><span class="p">,</span><span class="w"> </span><span class="n">loff_t</span><span class="w"> </span><span class="n">pos</span><span class="p">,</span><span class="w"> </span><span class="n">loff_t</span><span class="w"> </span><span class="n">length</span><span class="p">,</span>
<span class="w"> </span><span class="kt">unsigned</span><span class="w"> </span><span class="n">flags</span><span class="p">,</span><span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">iomap</span><span class="w"> </span><span class="o">*</span><span class="n">iomap</span><span class="p">,</span>
<span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">iomap</span><span class="w"> </span><span class="o">*</span><span class="n">srcmap</span><span class="p">);</span>
<span class="w"> </span><span class="kt">int</span><span class="w"> </span><span class="p">(</span><span class="o">*</span><span class="n">iomap_end</span><span class="p">)(</span><span class="k">struct</span><span class="w"> </span><span class="nc">inode</span><span class="w"> </span><span class="o">*</span><span class="n">inode</span><span class="p">,</span><span class="w"> </span><span class="n">loff_t</span><span class="w"> </span><span class="n">pos</span><span class="p">,</span><span class="w"> </span><span class="n">loff_t</span><span class="w"> </span><span class="n">length</span><span class="p">,</span>
<span class="w"> </span><span class="kt">ssize_t</span><span class="w"> </span><span class="n">written</span><span class="p">,</span><span class="w"> </span><span class="kt">unsigned</span><span class="w"> </span><span class="n">flags</span><span class="p">,</span>
<span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">iomap</span><span class="w"> </span><span class="o">*</span><span class="n">iomap</span><span class="p">);</span>
<span class="p">};</span>
</pre></div>
</div>
<section id="iomap-begin">
<h4><a class="toc-backref" href="#id8" role="doc-backlink"><span class="section-number">1.4.3.1. </span><code class="docutils literal notranslate"><span class="pre">->iomap_begin</span></code></a><a class="headerlink" href="#iomap-begin" title="Link to this heading">¶</a></h4>
<p>iomap operations call <code class="docutils literal notranslate"><span class="pre">->iomap_begin</span></code> to obtain one file mapping for
the range of bytes specified by <code class="docutils literal notranslate"><span class="pre">pos</span></code> and <code class="docutils literal notranslate"><span class="pre">length</span></code> for the file
<code class="docutils literal notranslate"><span class="pre">inode</span></code>.
This mapping should be returned through the <code class="docutils literal notranslate"><span class="pre">iomap</span></code> pointer.
The mapping must cover at least the first byte of the supplied file
range, but it does not need to cover the entire requested range.</p>
<p>Each iomap operation describes the requested operation through the
<code class="docutils literal notranslate"><span class="pre">flags</span></code> argument.
The exact value of <code class="docutils literal notranslate"><span class="pre">flags</span></code> will be documented in the
operation-specific sections below.
These flags can, at least in principle, apply generally to iomap
operations:</p>
<blockquote>
<div><ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">IOMAP_DIRECT</span></code> is set when the caller wishes to issue file I/O to
block storage.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">IOMAP_DAX</span></code> is set when the caller wishes to issue file I/O to
memory-like storage.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">IOMAP_NOWAIT</span></code> is set when the caller wishes to perform a best
effort attempt to avoid any operation that would result in blocking
the submitting task.
This is similar in intent to <code class="docutils literal notranslate"><span class="pre">O_NONBLOCK</span></code> for network APIs - it is
intended for asynchronous applications to keep doing other work
instead of waiting for the specific unavailable filesystem resource
to become available.
Filesystems implementing <code class="docutils literal notranslate"><span class="pre">IOMAP_NOWAIT</span></code> semantics need to use
trylock algorithms.
They need to be able to satisfy the entire I/O request range with a
single iomap mapping.
They need to avoid reading or writing metadata synchronously.
They need to avoid blocking memory allocations.
They need to avoid waiting on transaction reservations to allow
modifications to take place.
They probably should not be allocating new space.
And so on.
If there is any doubt in the filesystem developer’s mind as to
whether any specific <code class="docutils literal notranslate"><span class="pre">IOMAP_NOWAIT</span></code> operation may end up blocking,
then they should return <code class="docutils literal notranslate"><span class="pre">-EAGAIN</span></code> as early as possible rather than
start the operation and force the submitting task to block.
<code class="docutils literal notranslate"><span class="pre">IOMAP_NOWAIT</span></code> is often set on behalf of <code class="docutils literal notranslate"><span class="pre">IOCB_NOWAIT</span></code> or
<code class="docutils literal notranslate"><span class="pre">RWF_NOWAIT</span></code>.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">IOMAP_DONTCACHE</span></code> is set when the caller wishes to perform a
buffered file I/O and would like the kernel to drop the pagecache
after the I/O completes, if it isn’t already being used by another
thread.</p></li>
</ul>
</div></blockquote>
<p>If it is necessary to read existing file contents from a <a class="reference external" href="https://lore.kernel.org/all/20191008071527.29304-9-hch@lst.de/">different</a>
device or address range on a device, the filesystem should return that
information via <code class="docutils literal notranslate"><span class="pre">srcmap</span></code>.
Only pagecache and fsdax operations support reading from one mapping and
writing to another.</p>
</section>
<section id="iomap-end">
<h4><a class="toc-backref" href="#id9" role="doc-backlink"><span class="section-number">1.4.3.2. </span><code class="docutils literal notranslate"><span class="pre">->iomap_end</span></code></a><a class="headerlink" href="#iomap-end" title="Link to this heading">¶</a></h4>
<p>After the operation completes, the <code class="docutils literal notranslate"><span class="pre">->iomap_end</span></code> function, if present,
is called to signal that iomap is finished with a mapping.
Typically, implementations will use this function to tear down any
context that were set up in <code class="docutils literal notranslate"><span class="pre">->iomap_begin</span></code>.
For example, a write might wish to commit the reservations for the bytes
that were operated upon and unreserve any space that was not operated
upon.
<code class="docutils literal notranslate"><span class="pre">written</span></code> might be zero if no bytes were touched.
<code class="docutils literal notranslate"><span class="pre">flags</span></code> will contain the same value passed to <code class="docutils literal notranslate"><span class="pre">->iomap_begin</span></code>.
iomap ops for reads are not likely to need to supply this function.</p>
<p>Both functions should return a negative errno code on error, or zero on
success.</p>
</section>
</section>
</section>
<section id="preparing-for-file-operations">
<h2><a class="toc-backref" href="#id10" role="doc-backlink"><span class="section-number">1.5. </span>Preparing for File Operations</a><a class="headerlink" href="#preparing-for-file-operations" title="Link to this heading">¶</a></h2>
<p>iomap only handles mapping and I/O.
Filesystems must still call out to the VFS to check input parameters
and file state before initiating an I/O operation.
It does not handle obtaining filesystem freeze protection, updating of
timestamps, stripping privileges, or access control.</p>
</section>
<section id="locking-hierarchy">
<h2><a class="toc-backref" href="#id11" role="doc-backlink"><span class="section-number">1.6. </span>Locking Hierarchy</a><a class="headerlink" href="#locking-hierarchy" title="Link to this heading">¶</a></h2>
<p>iomap requires that filesystems supply their own locking model.
There are three categories of synchronization primitives, as far as
iomap is concerned:</p>
<blockquote>
<div><ul class="simple">
<li><p>The <strong>upper</strong> level primitive is provided by the filesystem to
coordinate access to different iomap operations.
The exact primitive is specific to the filesystem and operation,
but is often a VFS inode, pagecache invalidation, or folio lock.
For example, a filesystem might take <code class="docutils literal notranslate"><span class="pre">i_rwsem</span></code> before calling
<code class="docutils literal notranslate"><span class="pre">iomap_file_buffered_write</span></code> and <code class="docutils literal notranslate"><span class="pre">iomap_file_unshare</span></code> to prevent
these two file operations from clobbering each other.
Pagecache writeback may lock a folio to prevent other threads from
accessing the folio until writeback is underway.</p>
<ul>
<li><p>The <strong>lower</strong> level primitive is taken by the filesystem in the
<code class="docutils literal notranslate"><span class="pre">->iomap_begin</span></code> and <code class="docutils literal notranslate"><span class="pre">->iomap_end</span></code> functions to coordinate
access to the file space mapping information.
The fields of the iomap object should be filled out while holding
this primitive.
The upper level synchronization primitive, if any, remains held
while acquiring the lower level synchronization primitive.
For example, XFS takes <code class="docutils literal notranslate"><span class="pre">ILOCK_EXCL</span></code> and ext4 takes <code class="docutils literal notranslate"><span class="pre">i_data_sem</span></code>
while sampling mappings.
Filesystems with immutable mapping information may not require
synchronization here.</p></li>
<li><p>The <strong>operation</strong> primitive is taken by an iomap operation to
coordinate access to its own internal data structures.
The upper level synchronization primitive, if any, remains held
while acquiring this primitive.
The lower level primitive is not held while acquiring this
primitive.
For example, pagecache write operations will obtain a file mapping,
then grab and lock a folio to copy new contents.
It may also lock an internal folio state object to update metadata.</p></li>
</ul>
</li>
</ul>
</div></blockquote>
<p>The exact locking requirements are specific to the filesystem; for
certain operations, some of these locks can be elided.
All further mentions of locking are <em>recommendations</em>, not mandates.
Each filesystem author must figure out the locking for themself.</p>
</section>
<section id="bugs-and-limitations">
<h2><a class="toc-backref" href="#id12" role="doc-backlink"><span class="section-number">1.7. </span>Bugs and Limitations</a><a class="headerlink" href="#bugs-and-limitations" title="Link to this heading">¶</a></h2>
<blockquote>
<div><ul class="simple">
<li><p>No support for fscrypt.</p></li>
<li><p>No support for compression.</p></li>
<li><p>No support for fsverity yet.</p></li>
<li><p>Strong assumptions that IO should work the way it does on XFS.</p></li>
<li><p>Does iomap <em>actually</em> work for non-regular file data?</p></li>
</ul>
</div></blockquote>
<p>Patches welcome!</p>
</section>
</section>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©The kernel development community.
|
Powered by <a href="https://www.sphinx-doc.org/">Sphinx 8.1.3</a>
& <a href="https://alabaster.readthedocs.io">Alabaster 0.7.16</a>
|
<a href="../../_sources/filesystems/iomap/design.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>