<!DOCTYPE html>
<html lang="en" data-content_root="../">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>In-kernel memory-mapped I/O tracing — The Linux Kernel documentation</title>
<link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=fa44fd50" />
<link rel="stylesheet" type="text/css" href="../_static/alabaster.css?v=3918102e" />
<script src="../_static/documentation_options.js?v=5929fcd5"></script>
<script src="../_static/doctools.js?v=9bcbadda"></script>
<script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Hardware Latency Detector" href="hwlat_detector.html" />
<link rel="prev" title="HiSilicon PCIe Tune and Trace device" href="hisi-ptt.html" />
<link rel="stylesheet" href="../_static/custom.css" type="text/css" />
</head><body>
<div class="document">
<div class="sphinxsidebar" role="navigation" aria-label="Main">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../index.html">
<img class="logo" src="../_static/logo.svg" alt="Logo of The Linux Kernel"/>
</a></p>
<h1 class="logo"><a href="../index.html">The Linux Kernel</a></h1>
<p class="blurb">6.18.50</p>
<search id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</search>
<script>document.getElementById('searchbox').style.display = "block"</script>
<p>
<h3 class="kernel-toc-contents">Contents</h3>
<input type="checkbox" class="kernel-toc-toggle" id = "kernel-toc-toggle" checked>
<label class="kernel-toc-title" for="kernel-toc-toggle"></label>
<div class="kerneltoc" id="kerneltoc">
<ul>
<li class="toctree-l1"><a class="reference internal" href="../process/development-process.html">Development process</a></li>
<li class="toctree-l1"><a class="reference internal" href="../process/submitting-patches.html">Submitting patches</a></li>
<li class="toctree-l1"><a class="reference internal" href="../process/code-of-conduct.html">Code of conduct</a></li>
<li class="toctree-l1"><a class="reference internal" href="../maintainer/index.html">Maintainer handbook</a></li>
<li class="toctree-l1"><a class="reference internal" href="../process/index.html">All development-process docs</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../core-api/index.html">Core API</a></li>
<li class="toctree-l1"><a class="reference internal" href="../driver-api/index.html">Driver APIs</a></li>
<li class="toctree-l1"><a class="reference internal" href="../subsystem-apis.html">Subsystems</a></li>
<li class="toctree-l1"><a class="reference internal" href="../locking/index.html">Locking</a></li>
</ul>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../process/license-rules.html">Licensing rules</a></li>
<li class="toctree-l1"><a class="reference internal" href="../doc-guide/index.html">Writing documentation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../dev-tools/index.html">Development tools</a></li>
<li class="toctree-l1"><a class="reference internal" href="../dev-tools/testing-overview.html">Testing guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="../kernel-hacking/index.html">Hacking guide</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="index.html">Tracing</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="index.html#introduction-to-tracing">Introduction to Tracing</a></li>
<li class="toctree-l2"><a class="reference internal" href="index.html#core-tracing-frameworks">Core Tracing Frameworks</a></li>
<li class="toctree-l2"><a class="reference internal" href="index.html#event-tracing-and-analysis">Event Tracing and Analysis</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="index.html#hardware-and-performance-tracing">Hardware and Performance Tracing</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="intel_th.html">Intel(R) Trace Hub (TH)</a></li>
<li class="toctree-l3"><a class="reference internal" href="stm.html">System Trace Module</a></li>
<li class="toctree-l3"><a class="reference internal" href="sys-t.html">MIPI SyS-T over STP</a></li>
<li class="toctree-l3"><a class="reference internal" href="coresight/index.html">CoreSight - ARM Hardware Trace</a></li>
<li class="toctree-l3"><a class="reference internal" href="rv/index.html">Runtime Verification</a></li>
<li class="toctree-l3"><a class="reference internal" href="hisi-ptt.html">HiSilicon PCIe Tune and Trace device</a></li>
<li class="toctree-l3 current"><a class="current reference internal" href="#">In-kernel memory-mapped I/O tracing</a></li>
<li class="toctree-l3"><a class="reference internal" href="hwlat_detector.html">Hardware Latency Detector</a></li>
<li class="toctree-l3"><a class="reference internal" href="osnoise-tracer.html">OSNOISE Tracer</a></li>
<li class="toctree-l3"><a class="reference internal" href="timerlat-tracer.html">Timerlat tracer</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="index.html#user-space-tracing">User-Space Tracing</a></li>
<li class="toctree-l2"><a class="reference internal" href="index.html#additional-resources">Additional Resources</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../fault-injection/index.html">Fault injection</a></li>
<li class="toctree-l1"><a class="reference internal" href="../livepatch/index.html">Livepatching</a></li>
<li class="toctree-l1"><a class="reference internal" href="../rust/index.html">Rust</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../admin-guide/index.html">Administration</a></li>
<li class="toctree-l1"><a class="reference internal" href="../kbuild/index.html">Build system</a></li>
<li class="toctree-l1"><a class="reference internal" href="../admin-guide/reporting-issues.html">Reporting issues</a></li>
<li class="toctree-l1"><a class="reference internal" href="../tools/index.html">Userspace tools</a></li>
<li class="toctree-l1"><a class="reference internal" href="../userspace-api/index.html">Userspace API</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../firmware-guide/index.html">Firmware</a></li>
<li class="toctree-l1"><a class="reference internal" href="../devicetree/index.html">Firmware and Devicetree</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../arch/index.html">CPU architectures</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../staging/index.html">Unsorted documentation</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../translations/index.html">Translations</a></li>
</ul>
</div>
<script type="text/javascript"> <!--
var sbar = document.getElementsByClassName("sphinxsidebar")[0];
let currents = document.getElementsByClassName("current")
if (currents.length) {
sbar.scrollTop = currents[currents.length - 1].offsetTop;
}
--> </script>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/trace/mmiotrace.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
</div>
</div>
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="in-kernel-memory-mapped-i-o-tracing">
<h1>In-kernel memory-mapped I/O tracing<a class="headerlink" href="#in-kernel-memory-mapped-i-o-tracing" title="Link to this heading">¶</a></h1>
<p>Home page and links to optional user space tools:</p>
<blockquote>
<div><p><a class="reference external" href="https://nouveau.freedesktop.org/wiki/MmioTrace">https://nouveau.freedesktop.org/wiki/MmioTrace</a></p>
</div></blockquote>
<p>MMIO tracing was originally developed by Intel around 2003 for their Fault
Injection Test Harness. In Dec 2006 - Jan 2007, using the code from Intel,
Jeff Muizelaar created a tool for tracing MMIO accesses with the Nouveau
project in mind. Since then many people have contributed.</p>
<p>Mmiotrace was built for reverse engineering any memory-mapped IO device with
the Nouveau project as the first real user. Only x86 and x86_64 architectures
are supported.</p>
<p>Out-of-tree mmiotrace was originally modified for mainline inclusion and
ftrace framework by Pekka Paalanen <<a class="reference external" href="mailto:pq%40iki.fi">pq<span>@</span>iki<span>.</span>fi</a>>.</p>
<section id="preparation">
<h2>Preparation<a class="headerlink" href="#preparation" title="Link to this heading">¶</a></h2>
<p>Mmiotrace feature is compiled in by the CONFIG_MMIOTRACE option. Tracing is
disabled by default, so it is safe to have this set to yes. SMP systems are
supported, but tracing is unreliable and may miss events if more than one CPU
is on-line, therefore mmiotrace takes all but one CPU off-line during run-time
activation. You can re-enable CPUs by hand, but you have been warned, there
is no way to automatically detect if you are losing events due to CPUs racing.</p>
</section>
<section id="usage-quick-reference">
<h2>Usage Quick Reference<a class="headerlink" href="#usage-quick-reference" title="Link to this heading">¶</a></h2>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>$ mount -t debugfs debugfs /sys/kernel/debug
$ echo mmiotrace > /sys/kernel/tracing/current_tracer
$ cat /sys/kernel/tracing/trace_pipe > mydump.txt &
Start X or whatever.
$ echo "X is up" > /sys/kernel/tracing/trace_marker
$ echo nop > /sys/kernel/tracing/current_tracer
Check for lost events.
</pre></div>
</div>
</section>
<section id="usage">
<h2>Usage<a class="headerlink" href="#usage" title="Link to this heading">¶</a></h2>
<p>Make sure debugfs is mounted to /sys/kernel/debug.
If not (requires root privileges):</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>$ mount -t debugfs debugfs /sys/kernel/debug
</pre></div>
</div>
<p>Check that the driver you are about to trace is not loaded.</p>
<p>Activate mmiotrace (requires root privileges):</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>$ echo mmiotrace > /sys/kernel/tracing/current_tracer
</pre></div>
</div>
<p>Start storing the trace:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>$ cat /sys/kernel/tracing/trace_pipe > mydump.txt &
</pre></div>
</div>
<p>The ‘cat’ process should stay running (sleeping) in the background.</p>
<p>Load the driver you want to trace and use it. Mmiotrace will only catch MMIO
accesses to areas that are ioremapped while mmiotrace is active.</p>
<p>During tracing you can place comments (markers) into the trace by
$ echo “X is up” > /sys/kernel/tracing/trace_marker
This makes it easier to see which part of the (huge) trace corresponds to
which action. It is recommended to place descriptive markers about what you
do.</p>
<p>Shut down mmiotrace (requires root privileges):</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>$ echo nop > /sys/kernel/tracing/current_tracer
</pre></div>
</div>
<p>The ‘cat’ process exits. If it does not, kill it by issuing ‘fg’ command and
pressing ctrl+c.</p>
<p>Check that mmiotrace did not lose events due to a buffer filling up. Either:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>$ grep -i lost mydump.txt
</pre></div>
</div>
<p>which tells you exactly how many events were lost, or use:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>$ dmesg
</pre></div>
</div>
<p>to view your kernel log and look for “mmiotrace has lost events” warning. If
events were lost, the trace is incomplete. You should enlarge the buffers and
try again. Buffers are enlarged by first seeing how large the current buffers
are:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>$ cat /sys/kernel/tracing/buffer_size_kb
</pre></div>
</div>
<p>gives you a number. Approximately double this number and write it back, for
instance:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>$ echo 128000 > /sys/kernel/tracing/buffer_size_kb
</pre></div>
</div>
<p>Then start again from the top.</p>
<p>If you are doing a trace for a driver project, e.g. Nouveau, you should also
do the following before sending your results:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>$ lspci -vvv > lspci.txt
$ dmesg > dmesg.txt
$ tar zcf pciid-nick-mmiotrace.tar.gz mydump.txt lspci.txt dmesg.txt
</pre></div>
</div>
<p>and then send the .tar.gz file. The trace compresses considerably. Replace
“pciid” and “nick” with the PCI ID or model name of your piece of hardware
under investigation and your nickname.</p>
</section>
<section id="how-mmiotrace-works">
<h2>How Mmiotrace Works<a class="headerlink" href="#how-mmiotrace-works" title="Link to this heading">¶</a></h2>
<p>Access to hardware IO-memory is gained by mapping addresses from PCI bus by
calling one of the ioremap_*() functions. Mmiotrace is hooked into the
<code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">__ioremap()</span></code> function and gets called whenever a mapping is created. Mapping is
an event that is recorded into the trace log. Note that ISA range mappings
are not caught, since the mapping always exists and is returned directly.</p>
<p>MMIO accesses are recorded via page faults. Just before <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">__ioremap()</span></code> returns,
the mapped pages are marked as not present. Any access to the pages causes a
fault. The page fault handler calls mmiotrace to handle the fault. Mmiotrace
marks the page present, sets TF flag to achieve single stepping and exits the
fault handler. The instruction that faulted is executed and debug trap is
entered. Here mmiotrace again marks the page as not present. The instruction
is decoded to get the type of operation (read/write), data width and the value
read or written. These are stored to the trace log.</p>
<p>Setting the page present in the page fault handler has a race condition on SMP
machines. During the single stepping other CPUs may run freely on that page
and events can be missed without a notice. Re-enabling other CPUs during
tracing is discouraged.</p>
</section>
<section id="trace-log-format">
<h2>Trace Log Format<a class="headerlink" href="#trace-log-format" title="Link to this heading">¶</a></h2>
<p>The raw log is text and easily filtered with e.g. grep and awk. One record is
one line in the log. A record starts with a keyword, followed by keyword-
dependent arguments. Arguments are separated by a space, or continue until the
end of line. The format for version 20070824 is as follows:</p>
</section>
<section id="explanation-keyword-space-separated-arguments">
<h2>Explanation Keyword Space-separated arguments<a class="headerlink" href="#explanation-keyword-space-separated-arguments" title="Link to this heading">¶</a></h2>
<p>read event R width, timestamp, map id, physical, value, PC, PID
write event W width, timestamp, map id, physical, value, PC, PID
ioremap event MAP timestamp, map id, physical, virtual, length, PC, PID
iounmap event UNMAP timestamp, map id, PC, PID
marker MARK timestamp, text
version VERSION the string “20070824”
info for reader LSPCI one line from lspci -v
PCI address map PCIDEV space-separated /proc/bus/pci/devices data
unk. opcode UNKNOWN timestamp, map id, physical, data, PC, PID</p>
<p>Timestamp is in seconds with decimals. Physical is a PCI bus address, virtual
is a kernel virtual address. Width is the data width in bytes and value is the
data value. Map id is an arbitrary id number identifying the mapping that was
used in an operation. PC is the program counter and PID is process id. PC is
zero if it is not recorded. PID is always zero as tracing MMIO accesses
originating in user space memory is not yet supported.</p>
<p>For instance, the following awk filter will pass all 32-bit writes that target
physical addresses in the range [0xfb73ce40, 0xfb800000]</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>$ awk '/W 4 / { adr=strtonum($5); if (adr >= 0xfb73ce40 &&
adr < 0xfb800000) print; }'
</pre></div>
</div>
</section>
<section id="tools-for-developers">
<h2>Tools for Developers<a class="headerlink" href="#tools-for-developers" title="Link to this heading">¶</a></h2>
<dl class="simple">
<dt>The user space tools include utilities for:</dt><dd><ul class="simple">
<li><p>replacing numeric addresses and values with hardware register names</p></li>
<li><p>replaying MMIO logs, i.e., re-executing the recorded writes</p></li>
</ul>
</dd>
</dl>
</section>
</section>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©The kernel development community.
|
Powered by <a href="https://www.sphinx-doc.org/">Sphinx 8.1.3</a>
& <a href="https://alabaster.readthedocs.io">Alabaster 0.7.16</a>
|
<a href="../_sources/trace/mmiotrace.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>