<!DOCTYPE html>
<html lang="en" data-content_root="../">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Timestamping — The Linux Kernel documentation</title>
<link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=fa44fd50" />
<link rel="stylesheet" type="text/css" href="../_static/alabaster.css?v=3918102e" />
<script src="../_static/documentation_options.js?v=5929fcd5"></script>
<script src="../_static/doctools.js?v=9bcbadda"></script>
<script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Linux Kernel TIPC" href="tipc.html" />
<link rel="prev" title="Team" href="team.html" />
<link rel="stylesheet" href="../_static/custom.css" type="text/css" />
</head><body>
<div class="document">
<div class="sphinxsidebar" role="navigation" aria-label="Main">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../index.html">
<img class="logo" src="../_static/logo.svg" alt="Logo of The Linux Kernel"/>
</a></p>
<h1 class="logo"><a href="../index.html">The Linux Kernel</a></h1>
<p class="blurb">6.18.50</p>
<search id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</search>
<script>document.getElementById('searchbox').style.display = "block"</script>
<p>
<h3 class="kernel-toc-contents">Contents</h3>
<input type="checkbox" class="kernel-toc-toggle" id = "kernel-toc-toggle" checked>
<label class="kernel-toc-title" for="kernel-toc-toggle"></label>
<div class="kerneltoc" id="kerneltoc">
<ul>
<li class="toctree-l1"><a class="reference internal" href="../process/development-process.html">Development process</a></li>
<li class="toctree-l1"><a class="reference internal" href="../process/submitting-patches.html">Submitting patches</a></li>
<li class="toctree-l1"><a class="reference internal" href="../process/code-of-conduct.html">Code of conduct</a></li>
<li class="toctree-l1"><a class="reference internal" href="../maintainer/index.html">Maintainer handbook</a></li>
<li class="toctree-l1"><a class="reference internal" href="../process/index.html">All development-process docs</a></li>
</ul>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../core-api/index.html">Core API</a></li>
<li class="toctree-l1"><a class="reference internal" href="../driver-api/index.html">Driver APIs</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../subsystem-apis.html">Subsystems</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../subsystem-apis.html#core-subsystems">Core subsystems</a></li>
<li class="toctree-l2"><a class="reference internal" href="../subsystem-apis.html#human-interfaces">Human interfaces</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="../subsystem-apis.html#networking-interfaces">Networking interfaces</a><ul class="current">
<li class="toctree-l3 current"><a class="reference internal" href="index.html">Networking</a></li>
<li class="toctree-l3"><a class="reference internal" href="../netlabel/index.html">NetLabel</a></li>
<li class="toctree-l3"><a class="reference internal" href="../infiniband/index.html">InfiniBand</a></li>
<li class="toctree-l3"><a class="reference internal" href="../isdn/index.html">ISDN</a></li>
<li class="toctree-l3"><a class="reference internal" href="../mhi/index.html">MHI</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../subsystem-apis.html#storage-interfaces">Storage interfaces</a></li>
<li class="toctree-l2"><a class="reference internal" href="../subsystem-apis.html#other-subsystems">Other subsystems</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../locking/index.html">Locking</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../process/license-rules.html">Licensing rules</a></li>
<li class="toctree-l1"><a class="reference internal" href="../doc-guide/index.html">Writing documentation</a></li>
<li class="toctree-l1"><a class="reference internal" href="../dev-tools/index.html">Development tools</a></li>
<li class="toctree-l1"><a class="reference internal" href="../dev-tools/testing-overview.html">Testing guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="../kernel-hacking/index.html">Hacking guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="../trace/index.html">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../fault-injection/index.html">Fault injection</a></li>
<li class="toctree-l1"><a class="reference internal" href="../livepatch/index.html">Livepatching</a></li>
<li class="toctree-l1"><a class="reference internal" href="../rust/index.html">Rust</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../admin-guide/index.html">Administration</a></li>
<li class="toctree-l1"><a class="reference internal" href="../kbuild/index.html">Build system</a></li>
<li class="toctree-l1"><a class="reference internal" href="../admin-guide/reporting-issues.html">Reporting issues</a></li>
<li class="toctree-l1"><a class="reference internal" href="../tools/index.html">Userspace tools</a></li>
<li class="toctree-l1"><a class="reference internal" href="../userspace-api/index.html">Userspace API</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../firmware-guide/index.html">Firmware</a></li>
<li class="toctree-l1"><a class="reference internal" href="../devicetree/index.html">Firmware and Devicetree</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../arch/index.html">CPU architectures</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../staging/index.html">Unsorted documentation</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../translations/index.html">Translations</a></li>
</ul>
</div>
<script type="text/javascript"> <!--
var sbar = document.getElementsByClassName("sphinxsidebar")[0];
let currents = document.getElementsByClassName("current")
if (currents.length) {
sbar.scrollTop = currents[currents.length - 1].offsetTop;
}
--> </script>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/networking/timestamping.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
</div>
</div>
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="timestamping">
<h1>Timestamping<a class="headerlink" href="#timestamping" title="Link to this heading">¶</a></h1>
<section id="control-interfaces">
<h2>1. Control Interfaces<a class="headerlink" href="#control-interfaces" title="Link to this heading">¶</a></h2>
<p>The interfaces for receiving network packages timestamps are:</p>
<dl class="simple">
<dt>SO_TIMESTAMP</dt><dd><p>Generates a timestamp for each incoming packet in (not necessarily
monotonic) system time. Reports the timestamp via <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">recvmsg()</span></code> in a
control message in usec resolution.
SO_TIMESTAMP is defined as SO_TIMESTAMP_NEW or SO_TIMESTAMP_OLD
based on the architecture type and time_t representation of libc.
Control message format is in <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">__kernel_old_timeval</span></code> for
SO_TIMESTAMP_OLD and in <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">__kernel_sock_timeval</span></code> for
SO_TIMESTAMP_NEW options respectively.</p>
</dd>
<dt>SO_TIMESTAMPNS</dt><dd><p>Same timestamping mechanism as SO_TIMESTAMP, but reports the
timestamp as <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">timespec</span></code> in nsec resolution.
SO_TIMESTAMPNS is defined as SO_TIMESTAMPNS_NEW or SO_TIMESTAMPNS_OLD
based on the architecture type and time_t representation of libc.
Control message format is in <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">timespec</span></code> for SO_TIMESTAMPNS_OLD
and in <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">__kernel_timespec</span></code> for SO_TIMESTAMPNS_NEW options
respectively.</p>
</dd>
<dt>IP_MULTICAST_LOOP + SO_TIMESTAMP[NS]</dt><dd><p>Only for multicast:approximate transmit timestamp obtained by
reading the looped packet receive timestamp.</p>
</dd>
<dt>SO_TIMESTAMPING</dt><dd><p>Generates timestamps on reception, transmission or both. Supports
multiple timestamp sources, including hardware. Supports generating
timestamps for stream sockets.</p>
</dd>
</dl>
<section id="so-timestamp-also-so-timestamp-old-and-so-timestamp-new">
<h3>1.1 SO_TIMESTAMP (also SO_TIMESTAMP_OLD and SO_TIMESTAMP_NEW)<a class="headerlink" href="#so-timestamp-also-so-timestamp-old-and-so-timestamp-new" title="Link to this heading">¶</a></h3>
<p>This socket option enables timestamping of datagrams on the reception
path. Because the destination socket, if any, is not known early in
the network stack, the feature has to be enabled for all packets. The
same is true for all early receive timestamp options.</p>
<p>For interface details, see <cite>man 7 socket</cite>.</p>
<p>Always use SO_TIMESTAMP_NEW timestamp to always get timestamp in
<code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">__kernel_sock_timeval</span></code> format.</p>
<p>SO_TIMESTAMP_OLD returns incorrect timestamps after the year 2038
on 32 bit machines.</p>
</section>
<section id="so-timestampns-also-so-timestampns-old-and-so-timestampns-new">
<h3>1.2 SO_TIMESTAMPNS (also SO_TIMESTAMPNS_OLD and SO_TIMESTAMPNS_NEW)<a class="headerlink" href="#so-timestampns-also-so-timestampns-old-and-so-timestampns-new" title="Link to this heading">¶</a></h3>
<p>This option is identical to SO_TIMESTAMP except for the returned data type.
Its <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">timespec</span></code> allows for higher resolution (ns) timestamps than the
timeval of SO_TIMESTAMP (ms).</p>
<p>Always use SO_TIMESTAMPNS_NEW timestamp to always get timestamp in
<code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">__kernel_timespec</span></code> format.</p>
<p>SO_TIMESTAMPNS_OLD returns incorrect timestamps after the year 2038
on 32 bit machines.</p>
</section>
<section id="so-timestamping-also-so-timestamping-old-and-so-timestamping-new">
<h3>1.3 SO_TIMESTAMPING (also SO_TIMESTAMPING_OLD and SO_TIMESTAMPING_NEW)<a class="headerlink" href="#so-timestamping-also-so-timestamping-old-and-so-timestamping-new" title="Link to this heading">¶</a></h3>
<p>Supports multiple types of timestamp requests. As a result, this
socket option takes a bitmap of flags, not a boolean. In:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>err = setsockopt(fd, SOL_SOCKET, SO_TIMESTAMPING, &val, sizeof(val));
</pre></div>
</div>
<p>val is an integer with any of the following bits set. Setting other
bit returns EINVAL and does not change the current state.</p>
<p>The socket option configures timestamp generation for individual
sk_buffs (1.3.1), timestamp reporting to the socket’s error
queue (1.3.2) and options (1.3.3). Timestamp generation can also
be enabled for individual sendmsg calls using cmsg (1.3.4).</p>
<section id="timestamp-generation">
<h4>1.3.1 Timestamp Generation<a class="headerlink" href="#timestamp-generation" title="Link to this heading">¶</a></h4>
<p>Some bits are requests to the stack to try to generate timestamps. Any
combination of them is valid. Changes to these bits apply to newly
created packets, not to packets already in the stack. As a result, it
is possible to selectively request timestamps for a subset of packets
(e.g., for sampling) by embedding an <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">send()</span></code> call within two setsockopt
calls, one to enable timestamp generation and one to disable it.
Timestamps may also be generated for reasons other than being
requested by a particular socket, such as when receive timestamping is
enabled system wide, as explained earlier.</p>
<dl class="simple">
<dt>SOF_TIMESTAMPING_RX_HARDWARE:</dt><dd><p>Request rx timestamps generated by the network adapter.</p>
</dd>
<dt>SOF_TIMESTAMPING_RX_SOFTWARE:</dt><dd><p>Request rx timestamps when data enters the kernel. These timestamps
are generated just after a device driver hands a packet to the
kernel receive stack.</p>
</dd>
<dt>SOF_TIMESTAMPING_TX_HARDWARE:</dt><dd><p>Request tx timestamps generated by the network adapter. This flag
can be enabled via both socket options and control messages.</p>
</dd>
<dt>SOF_TIMESTAMPING_TX_SOFTWARE:</dt><dd><p>Request tx timestamps when data leaves the kernel. These timestamps
are generated in the device driver as close as possible, but always
prior to, passing the packet to the network interface. Hence, they
require driver support and may not be available for all devices.
This flag can be enabled via both socket options and control messages.</p>
</dd>
<dt>SOF_TIMESTAMPING_TX_SCHED:</dt><dd><p>Request tx timestamps prior to entering the packet scheduler. Kernel
transmit latency is, if long, often dominated by queuing delay. The
difference between this timestamp and one taken at
SOF_TIMESTAMPING_TX_SOFTWARE will expose this latency independent
of protocol processing. The latency incurred in protocol
processing, if any, can be computed by subtracting a userspace
timestamp taken immediately before <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">send()</span></code> from this timestamp. On
machines with virtual devices where a transmitted packet travels
through multiple devices and, hence, multiple packet schedulers,
a timestamp is generated at each layer. This allows for fine
grained measurement of queuing delay. This flag can be enabled
via both socket options and control messages.</p>
</dd>
<dt>SOF_TIMESTAMPING_TX_ACK:</dt><dd><p>Request tx timestamps when all data in the send buffer has been
acknowledged. This only makes sense for reliable protocols. It is
currently only implemented for TCP. For that protocol, it may
over-report measurement, because the timestamp is generated when all
data up to and including the buffer at <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">send()</span></code> was acknowledged: the
cumulative acknowledgment. The mechanism ignores SACK and FACK.
This flag can be enabled via both socket options and control messages.</p>
</dd>
<dt>SOF_TIMESTAMPING_TX_COMPLETION:</dt><dd><p>Request tx timestamps on packet tx completion. The completion
timestamp is generated by the kernel when it receives packet a
completion report from the hardware. Hardware may report multiple
packets at once, and completion timestamps reflect the timing of the
report and not actual tx time. This flag can be enabled via both
socket options and control messages.</p>
</dd>
</dl>
</section>
<section id="timestamp-reporting">
<h4>1.3.2 Timestamp Reporting<a class="headerlink" href="#timestamp-reporting" title="Link to this heading">¶</a></h4>
<p>The other three bits control which timestamps will be reported in a
generated control message. Changes to the bits take immediate
effect at the timestamp reporting locations in the stack. Timestamps
are only reported for packets that also have the relevant timestamp
generation request set.</p>
<dl class="simple">
<dt>SOF_TIMESTAMPING_SOFTWARE:</dt><dd><p>Report any software timestamps when available.</p>
</dd>
<dt>SOF_TIMESTAMPING_SYS_HARDWARE:</dt><dd><p>This option is deprecated and ignored.</p>
</dd>
<dt>SOF_TIMESTAMPING_RAW_HARDWARE:</dt><dd><p>Report hardware timestamps as generated by
SOF_TIMESTAMPING_TX_HARDWARE or SOF_TIMESTAMPING_RX_HARDWARE
when available.</p>
</dd>
</dl>
</section>
<section id="timestamp-options">
<h4>1.3.3 Timestamp Options<a class="headerlink" href="#timestamp-options" title="Link to this heading">¶</a></h4>
<p>The interface supports the options</p>
<dl>
<dt>SOF_TIMESTAMPING_OPT_ID:</dt><dd><p>Generate a unique identifier along with each packet. A process can
have multiple concurrent timestamping requests outstanding. Packets
can be reordered in the transmit path, for instance in the packet
scheduler. In that case timestamps will be queued onto the error
queue out of order from the original <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">send()</span></code> calls. It is not always
possible to uniquely match timestamps to the original <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">send()</span></code> calls
based on timestamp order or payload inspection alone, then.</p>
<p>This option associates each packet at <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">send()</span></code> with a unique
identifier and returns that along with the timestamp. The identifier
is derived from a per-socket u32 counter (that wraps). For datagram
sockets, the counter increments with each sent packet. For stream
sockets, it increments with every byte. For stream sockets, also set
SOF_TIMESTAMPING_OPT_ID_TCP, see the section below.</p>
<p>The counter starts at zero. It is initialized the first time that
the socket option is enabled. It is reset each time the option is
enabled after having been disabled. Resetting the counter does not
change the identifiers of existing packets in the system.</p>
<p>This option is implemented only for transmit timestamps. There, the
timestamp is always looped along with a <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">sock_extended_err</span></code>.
The option modifies field ee_data to pass an id that is unique
among all possibly concurrently outstanding timestamp requests for
that socket.</p>
<p>The process can optionally override the default generated ID, by
passing a specific ID with control message SCM_TS_OPT_ID (not
supported for TCP sockets):</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>struct msghdr *msg;
...
cmsg = CMSG_FIRSTHDR(msg);
cmsg->cmsg_level = SOL_SOCKET;
cmsg->cmsg_type = SCM_TS_OPT_ID;
cmsg->cmsg_len = CMSG_LEN(sizeof(__u32));
*((__u32 *) CMSG_DATA(cmsg)) = opt_id;
err = sendmsg(fd, msg, 0);
</pre></div>
</div>
</dd>
<dt>SOF_TIMESTAMPING_OPT_ID_TCP:</dt><dd><p>Pass this modifier along with SOF_TIMESTAMPING_OPT_ID for new TCP
timestamping applications. SOF_TIMESTAMPING_OPT_ID defines how the
counter increments for stream sockets, but its starting point is
not entirely trivial. This option fixes that.</p>
<p>For stream sockets, if SOF_TIMESTAMPING_OPT_ID is set, this should
always be set too. On datagram sockets the option has no effect.</p>
<p>A reasonable expectation is that the counter is reset to zero with
the system call, so that a subsequent write() of N bytes generates
a timestamp with counter N-1. SOF_TIMESTAMPING_OPT_ID_TCP
implements this behavior under all conditions.</p>
<p>SOF_TIMESTAMPING_OPT_ID without modifier often reports the same,
especially when the socket option is set when no data is in
transmission. If data is being transmitted, it may be off by the
length of the output queue (SIOCOUTQ).</p>
<p>The difference is due to being based on snd_una versus write_seq.
snd_una is the offset in the stream acknowledged by the peer. This
depends on factors outside of process control, such as network RTT.
write_seq is the last byte written by the process. This offset is
not affected by external inputs.</p>
<p>The difference is subtle and unlikely to be noticed when configured
at initial socket creation, when no data is queued or sent. But
SOF_TIMESTAMPING_OPT_ID_TCP behavior is more robust regardless of
when the socket option is set.</p>
</dd>
<dt>SOF_TIMESTAMPING_OPT_CMSG:</dt><dd><p>Support <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">recv()</span></code> cmsg for all timestamped packets. Control messages
are already supported unconditionally on all packets with receive
timestamps and on IPv6 packets with transmit timestamp. This option
extends them to IPv4 packets with transmit timestamp. One use case
is to correlate packets with their egress device, by enabling socket
option IP_PKTINFO simultaneously.</p>
</dd>
<dt>SOF_TIMESTAMPING_OPT_TSONLY:</dt><dd><p>Applies to transmit timestamps only. Makes the kernel return the
timestamp as a cmsg alongside an empty packet, as opposed to
alongside the original packet. This reduces the amount of memory
charged to the socket’s receive budget (SO_RCVBUF) and delivers
the timestamp even if sysctl net.core.tstamp_allow_data is 0.
This option disables SOF_TIMESTAMPING_OPT_CMSG.</p>
</dd>
<dt>SOF_TIMESTAMPING_OPT_STATS:</dt><dd><p>Optional stats that are obtained along with the transmit timestamps.
It must be used together with SOF_TIMESTAMPING_OPT_TSONLY. When the
transmit timestamp is available, the stats are available in a
separate control message of type SCM_TIMESTAMPING_OPT_STATS, as a
list of TLVs (<code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">nlattr</span></code>) of types. These stats allow the
application to associate various transport layer stats with
the transmit timestamps, such as how long a certain block of
data was limited by peer’s receiver window.</p>
</dd>
<dt>SOF_TIMESTAMPING_OPT_PKTINFO:</dt><dd><p>Enable the SCM_TIMESTAMPING_PKTINFO control message for incoming
packets with hardware timestamps. The message contains <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span>
<span class="pre">scm_ts_pktinfo</span></code>, which supplies the index of the real interface which
received the packet and its length at layer 2. A valid (non-zero)
interface index will be returned only if CONFIG_NET_RX_BUSY_POLL is
enabled and the driver is using NAPI. The <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">contains</span></code> also two
other fields, but they are reserved and undefined.</p>
</dd>
<dt>SOF_TIMESTAMPING_OPT_TX_SWHW:</dt><dd><p>Request both hardware and software timestamps for outgoing packets
when SOF_TIMESTAMPING_TX_HARDWARE and SOF_TIMESTAMPING_TX_SOFTWARE
are enabled at the same time. If both timestamps are generated,
two separate messages will be looped to the socket’s error queue,
each containing just one timestamp.</p>
</dd>
<dt>SOF_TIMESTAMPING_OPT_RX_FILTER:</dt><dd><p>Filter out spurious receive timestamps: report a receive timestamp
only if the matching timestamp generation flag is enabled.</p>
<p>Receive timestamps are generated early in the ingress path, before a
packet’s destination socket is known. If any socket enables receive
timestamps, packets for all socket will receive timestamped packets.
Including those that request timestamp reporting with
SOF_TIMESTAMPING_SOFTWARE and/or SOF_TIMESTAMPING_RAW_HARDWARE, but
do not request receive timestamp generation. This can happen when
requesting transmit timestamps only.</p>
<p>Receiving spurious timestamps is generally benign. A process can
ignore the unexpected non-zero value. But it makes behavior subtly
dependent on other sockets. This flag isolates the socket for more
deterministic behavior.</p>
</dd>
</dl>
<p>New applications are encouraged to pass SOF_TIMESTAMPING_OPT_ID to
disambiguate timestamps and SOF_TIMESTAMPING_OPT_TSONLY to operate
regardless of the setting of sysctl net.core.tstamp_allow_data.</p>
<p>An exception is when a process needs additional cmsg data, for
instance SOL_IP/IP_PKTINFO to detect the egress network interface.
Then pass option SOF_TIMESTAMPING_OPT_CMSG. This option depends on
having access to the contents of the original packet, so cannot be
combined with SOF_TIMESTAMPING_OPT_TSONLY.</p>
</section>
<section id="enabling-timestamps-via-control-messages">
<h4>1.3.4. Enabling timestamps via control messages<a class="headerlink" href="#enabling-timestamps-via-control-messages" title="Link to this heading">¶</a></h4>
<p>In addition to socket options, timestamp generation can be requested
per write via cmsg, only for SOF_TIMESTAMPING_TX_* (see Section 1.3.1).
Using this feature, applications can sample timestamps per <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">sendmsg()</span></code>
without paying the overhead of enabling and disabling timestamps via
setsockopt:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>struct msghdr *msg;
...
cmsg = CMSG_FIRSTHDR(msg);
cmsg->cmsg_level = SOL_SOCKET;
cmsg->cmsg_type = SO_TIMESTAMPING;
cmsg->cmsg_len = CMSG_LEN(sizeof(__u32));
*((__u32 *) CMSG_DATA(cmsg)) = SOF_TIMESTAMPING_TX_SCHED |
SOF_TIMESTAMPING_TX_SOFTWARE |
SOF_TIMESTAMPING_TX_ACK;
err = sendmsg(fd, msg, 0);
</pre></div>
</div>
<p>The SOF_TIMESTAMPING_TX_* flags set via cmsg will override
the SOF_TIMESTAMPING_TX_* flags set via setsockopt.</p>
<p>Moreover, applications must still enable timestamp reporting via
setsockopt to receive timestamps:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>__u32 val = SOF_TIMESTAMPING_SOFTWARE |
SOF_TIMESTAMPING_OPT_ID /* or any other flag */;
err = setsockopt(fd, SOL_SOCKET, SO_TIMESTAMPING, &val, sizeof(val));
</pre></div>
</div>
</section>
</section>
<section id="bytestream-timestamps">
<h3>1.4 Bytestream Timestamps<a class="headerlink" href="#bytestream-timestamps" title="Link to this heading">¶</a></h3>
<p>The SO_TIMESTAMPING interface supports timestamping of bytes in a
bytestream. Each request is interpreted as a request for when the
entire contents of the buffer has passed a timestamping point. That
is, for streams option SOF_TIMESTAMPING_TX_SOFTWARE will record
when all bytes have reached the device driver, regardless of how
many packets the data has been converted into.</p>
<p>In general, bytestreams have no natural delimiters and therefore
correlating a timestamp with data is non-trivial. A range of bytes
may be split across segments, any segments may be merged (possibly
coalescing sections of previously segmented buffers associated with
independent <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">send()</span></code> calls). Segments can be reordered and the same
byte range can coexist in multiple segments for protocols that
implement retransmissions.</p>
<p>It is essential that all timestamps implement the same semantics,
regardless of these possible transformations, as otherwise they are
incomparable. Handling “rare” corner cases differently from the
simple case (a 1:1 mapping from buffer to skb) is insufficient
because performance debugging often needs to focus on such outliers.</p>
<p>In practice, timestamps can be correlated with segments of a
bytestream consistently, if both semantics of the timestamp and the
timing of measurement are chosen correctly. This challenge is no
different from deciding on a strategy for IP fragmentation. There, the
definition is that only the first fragment is timestamped. For
bytestreams, we chose that a timestamp is generated only when all
bytes have passed a point. SOF_TIMESTAMPING_TX_ACK as defined is easy to
implement and reason about. An implementation that has to take into
account SACK would be more complex due to possible transmission holes
and out of order arrival.</p>
<p>On the host, TCP can also break the simple 1:1 mapping from buffer to
skbuff as a result of Nagle, cork, autocork, segmentation and GSO. The
implementation ensures correctness in all cases by tracking the
individual last byte passed to <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">send()</span></code>, even if it is no longer the
last byte after an skbuff extend or merge operation. It stores the
relevant sequence number in skb_shinfo(skb)->tskey. Because an skbuff
has only one such field, only one timestamp can be generated.</p>
<p>In rare cases, a timestamp request can be missed if two requests are
collapsed onto the same skb. A process can detect this situation by
enabling SOF_TIMESTAMPING_OPT_ID and comparing the byte offset at
send time with the value returned for each timestamp. It can prevent
the situation by always flushing the TCP stack in between requests,
for instance by enabling TCP_NODELAY and disabling TCP_CORK and
autocork. After linux-4.7, a better way to prevent coalescing is
to use MSG_EOR flag at <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">sendmsg()</span></code> time.</p>
<p>These precautions ensure that the timestamp is generated only when all
bytes have passed a timestamp point, assuming that the network stack
itself does not reorder the segments. The stack indeed tries to avoid
reordering. The one exception is under administrator control: it is
possible to construct a packet scheduler configuration that delays
segments from the same stream differently. Such a setup would be
unusual.</p>
</section>
</section>
<section id="data-interfaces">
<h2>2 Data Interfaces<a class="headerlink" href="#data-interfaces" title="Link to this heading">¶</a></h2>
<p>Timestamps are read using the ancillary data feature of <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">recvmsg()</span></code>.
See <cite>man 3 cmsg</cite> for details of this interface. The socket manual
page (<cite>man 7 socket</cite>) describes how timestamps generated with
SO_TIMESTAMP and SO_TIMESTAMPNS records can be retrieved.</p>
<section id="scm-timestamping-records">
<h3>2.1 SCM_TIMESTAMPING records<a class="headerlink" href="#scm-timestamping-records" title="Link to this heading">¶</a></h3>
<p>These timestamps are returned in a control message with cmsg_level
SOL_SOCKET, cmsg_type SCM_TIMESTAMPING, and payload of type</p>
<p>For SO_TIMESTAMPING_OLD:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>struct scm_timestamping {
struct timespec ts[3];
};
</pre></div>
</div>
<p>For SO_TIMESTAMPING_NEW:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>struct scm_timestamping64 {
struct __kernel_timespec ts[3];
</pre></div>
</div>
<p>Always use SO_TIMESTAMPING_NEW timestamp to always get timestamp in
<code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">scm_timestamping64</span></code> format.</p>
<p>SO_TIMESTAMPING_OLD returns incorrect timestamps after the year 2038
on 32 bit machines.</p>
<p>The structure can return up to three timestamps. This is a legacy
feature. At least one field is non-zero at any time. Most timestamps
are passed in ts[0]. Hardware timestamps are passed in ts[2].</p>
<p>ts[1] used to hold hardware timestamps converted to system time.
Instead, expose the hardware clock device on the NIC directly as
a HW PTP clock source, to allow time conversion in userspace and
optionally synchronize system time with a userspace PTP stack such
as linuxptp. For the PTP clock API, see <a class="reference internal" href="../driver-api/ptp.html"><span class="doc">PTP hardware clock infrastructure for Linux</span></a>.</p>
<p>Note that if the SO_TIMESTAMP or SO_TIMESTAMPNS option is enabled
together with SO_TIMESTAMPING using SOF_TIMESTAMPING_SOFTWARE, a false
software timestamp will be generated in the <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">recvmsg()</span></code> call and passed
in ts[0] when a real software timestamp is missing. This happens also
on hardware transmit timestamps.</p>
<section id="transmit-timestamps-with-msg-errqueue">
<h4>2.1.1 Transmit timestamps with MSG_ERRQUEUE<a class="headerlink" href="#transmit-timestamps-with-msg-errqueue" title="Link to this heading">¶</a></h4>
<p>For transmit timestamps the outgoing packet is looped back to the
socket’s error queue with the send timestamp(s) attached. A process
receives the timestamps by calling <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">recvmsg()</span></code> with flag MSG_ERRQUEUE
set and with a msg_control buffer sufficiently large to receive the
relevant metadata structures. The recvmsg call returns the original
outgoing data packet with two ancillary messages attached.</p>
<p>A message of cm_level SOL_IP(V6) and cm_type IP(V6)_RECVERR
embeds a <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">sock_extended_err</span></code>. This defines the error type. For
timestamps, the ee_errno field is ENOMSG. The other ancillary message
will have cm_level SOL_SOCKET and cm_type SCM_TIMESTAMPING. This
embeds the <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">scm_timestamping</span></code>.</p>
<section id="timestamp-types">
<h5>2.1.1.2 Timestamp types<a class="headerlink" href="#timestamp-types" title="Link to this heading">¶</a></h5>
<p>The semantics of the three <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">timespec</span></code> are defined by field
ee_info in the extended error structure. It contains a value of
type SCM_TSTAMP_* to define the actual timestamp passed in
scm_timestamping.</p>
<p>The SCM_TSTAMP_* types are 1:1 matches to the SOF_TIMESTAMPING_*
control fields discussed previously, with one exception. For legacy
reasons, SCM_TSTAMP_SND is equal to zero and can be set for both
SOF_TIMESTAMPING_TX_HARDWARE and SOF_TIMESTAMPING_TX_SOFTWARE. It
is the first if ts[2] is non-zero, the second otherwise, in which
case the timestamp is stored in ts[0].</p>
</section>
<section id="fragmentation">
<h5>2.1.1.3 Fragmentation<a class="headerlink" href="#fragmentation" title="Link to this heading">¶</a></h5>
<p>Fragmentation of outgoing datagrams is rare, but is possible, e.g., by
explicitly disabling PMTU discovery. If an outgoing packet is fragmented,
then only the first fragment is timestamped and returned to the sending
socket.</p>
</section>
<section id="packet-payload">
<h5>2.1.1.4 Packet Payload<a class="headerlink" href="#packet-payload" title="Link to this heading">¶</a></h5>
<p>The calling application is often not interested in receiving the whole
packet payload that it passed to the stack originally: the socket
error queue mechanism is just a method to piggyback the timestamp on.
In this case, the application can choose to read datagrams with a
smaller buffer, possibly even of length 0. The payload is truncated
accordingly. Until the process calls <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">recvmsg()</span></code> on the error queue,
however, the full packet is queued, taking up budget from SO_RCVBUF.</p>
</section>
<section id="blocking-read">
<h5>2.1.1.5 Blocking Read<a class="headerlink" href="#blocking-read" title="Link to this heading">¶</a></h5>
<p>Reading from the error queue is always a non-blocking operation. To
block waiting on a timestamp, use poll or select. poll() will return
POLLERR in pollfd.revents if any data is ready on the error queue.
There is no need to pass this flag in pollfd.events. This flag is
ignored on request. See also <cite>man 2 poll</cite>.</p>
</section>
</section>
<section id="receive-timestamps">
<h4>2.1.2 Receive timestamps<a class="headerlink" href="#receive-timestamps" title="Link to this heading">¶</a></h4>
<p>On reception, there is no reason to read from the socket error queue.
The SCM_TIMESTAMPING ancillary data is sent along with the packet data
on a normal <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">recvmsg()</span></code>. Since this is not a socket error, it is not
accompanied by a message SOL_IP(V6)/IP(V6)_RECVERROR. In this case,
the meaning of the three fields in <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">scm_timestamping</span></code> is
implicitly defined. ts[0] holds a software timestamp if set, ts[1]
is again deprecated and ts[2] holds a hardware timestamp if set.</p>
</section>
</section>
</section>
<section id="hardware-timestamping-configuration-ethtool-msg-tsconfig-set-get">
<h2>3. Hardware Timestamping configuration: ETHTOOL_MSG_TSCONFIG_SET/GET<a class="headerlink" href="#hardware-timestamping-configuration-ethtool-msg-tsconfig-set-get" title="Link to this heading">¶</a></h2>
<p>Hardware time stamping must also be initialized for each device driver
that is expected to do hardware time stamping. The parameter is defined in
include/uapi/linux/net_tstamp.h as:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>struct hwtstamp_config {
int flags; /* no flags defined right now, must be zero */
int tx_type; /* HWTSTAMP_TX_* */
int rx_filter; /* HWTSTAMP_FILTER_* */
};
</pre></div>
</div>
<p>Desired behavior is passed into the kernel and to a specific device by
calling the tsconfig netlink socket <code class="docutils literal notranslate"><span class="pre">ETHTOOL_MSG_TSCONFIG_SET</span></code>.
The <code class="docutils literal notranslate"><span class="pre">ETHTOOL_A_TSCONFIG_TX_TYPES</span></code>, <code class="docutils literal notranslate"><span class="pre">ETHTOOL_A_TSCONFIG_RX_FILTERS</span></code> and
<code class="docutils literal notranslate"><span class="pre">ETHTOOL_A_TSCONFIG_HWTSTAMP_FLAGS</span></code> netlink attributes are then used to set
the <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">hwtstamp_config</span></code> accordingly.</p>
<p>The <code class="docutils literal notranslate"><span class="pre">ETHTOOL_A_TSCONFIG_HWTSTAMP_PROVIDER</span></code> netlink nested attribute is used
to select the source of the hardware time stamping. It is composed of an index
for the device source and a qualifier for the type of time stamping.</p>
<p>Drivers are free to use a more permissive configuration than the requested
configuration. It is expected that drivers should only implement directly the
most generic mode that can be supported. For example if the hardware can
support HWTSTAMP_FILTER_PTP_V2_EVENT, then it should generally always upscale
HWTSTAMP_FILTER_PTP_V2_L2_SYNC, and so forth, as HWTSTAMP_FILTER_PTP_V2_EVENT
is more generic (and more useful to applications).</p>
<p>A driver which supports hardware time stamping shall update the <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span>
<span class="pre">with</span></code> the actual, possibly more permissive configuration. If the
requested packets cannot be time stamped, then nothing should be
changed and ERANGE shall be returned (in contrast to EINVAL, which
indicates that SIOCSHWTSTAMP is not supported at all).</p>
<p>Only a processes with admin rights may change the configuration. User
space is responsible to ensure that multiple processes don’t interfere
with each other and that the settings are reset.</p>
<p>Any process can read the actual configuration by requesting tsconfig netlink
socket <code class="docutils literal notranslate"><span class="pre">ETHTOOL_MSG_TSCONFIG_GET</span></code>.</p>
<p>The legacy configuration is the use of the ioctl(SIOCSHWTSTAMP) with a pointer
to a <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">ifreq</span></code> whose ifr_data points to a <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">hwtstamp_config</span></code>.
The tx_type and rx_filter are hints to the driver what it is expected to do.
If the requested fine-grained filtering for incoming packets is not
supported, the driver may time stamp more than just the requested types
of packets. ioctl(SIOCGHWTSTAMP) is used in the same way as the
ioctl(SIOCSHWTSTAMP). However, this has not been implemented in all drivers.</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>/* possible values for hwtstamp_config->tx_type */
enum {
/*
* no outgoing packet will need hardware time stamping;
* should a packet arrive which asks for it, no hardware
* time stamping will be done
*/
HWTSTAMP_TX_OFF,
/*
* enables hardware time stamping for outgoing packets;
* the sender of the packet decides which are to be
* time stamped by setting SOF_TIMESTAMPING_TX_SOFTWARE
* before sending the packet
*/
HWTSTAMP_TX_ON,
};
/* possible values for hwtstamp_config->rx_filter */
enum {
/* time stamp no incoming packet at all */
HWTSTAMP_FILTER_NONE,
/* time stamp any incoming packet */
HWTSTAMP_FILTER_ALL,
/* return value: time stamp all packets requested plus some others */
HWTSTAMP_FILTER_SOME,
/* PTP v1, UDP, any kind of event packet */
HWTSTAMP_FILTER_PTP_V1_L4_EVENT,
/* for the complete list of values, please check
* the include file include/uapi/linux/net_tstamp.h
*/
};
</pre></div>
</div>
<section id="hardware-timestamping-implementation-device-drivers">
<h3>3.1 Hardware Timestamping Implementation: Device Drivers<a class="headerlink" href="#hardware-timestamping-implementation-device-drivers" title="Link to this heading">¶</a></h3>
<p>A driver which supports hardware time stamping must support the
ndo_hwtstamp_set NDO or the legacy SIOCSHWTSTAMP ioctl and update the
supplied <code class="xref c c-struct broken_xref docutils literal notranslate"><span class="pre">struct</span> <span class="pre">hwtstamp_config</span></code> with the actual values as described in
the section on SIOCSHWTSTAMP. It should also support ndo_hwtstamp_get or
the legacy SIOCGHWTSTAMP.</p>
<p>Time stamps for received packets must be stored in the skb. To get a pointer
to the shared time stamp structure of the skb call <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">skb_hwtstamps()</span></code>. Then
set the time stamps in the structure:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>struct skb_shared_hwtstamps {
/* hardware time stamp transformed into duration
* since arbitrary point in time
*/
ktime_t hwtstamp;
};
</pre></div>
</div>
<p>Time stamps for outgoing packets are to be generated as follows:</p>
<ul>
<li><p>In <code class="xref c c-func broken_xref docutils literal notranslate"><span class="pre">hard_start_xmit()</span></code>, check if (skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP)
is set no-zero. If yes, then the driver is expected to do hardware time
stamping.</p></li>
<li><p>If this is possible for the skb and requested, then declare
that the driver is doing the time stamping by setting the flag
SKBTX_IN_PROGRESS in skb_shinfo(skb)->tx_flags , e.g. with:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
</pre></div>
</div>
<p>You might want to keep a pointer to the associated skb for the next step
and not free the skb. A driver not supporting hardware time stamping doesn’t
do that. A driver must never touch sk_buff::tstamp! It is used to store
software generated time stamps by the network subsystem.</p>
</li>
<li><p>Driver should call <a class="reference internal" href="kapi.html#c.skb_tx_timestamp" title="skb_tx_timestamp"><code class="xref c c-func docutils literal notranslate"><span class="pre">skb_tx_timestamp()</span></code></a> as close to passing sk_buff to hardware
as possible. <a class="reference internal" href="kapi.html#c.skb_tx_timestamp" title="skb_tx_timestamp"><code class="xref c c-func docutils literal notranslate"><span class="pre">skb_tx_timestamp()</span></code></a> provides a software time stamp if requested
and hardware timestamping is not possible (SKBTX_IN_PROGRESS not set).</p></li>
<li><p>As soon as the driver has sent the packet and/or obtained a
hardware time stamp for it, it passes the time stamp back by
calling <a class="reference internal" href="kapi.html#c.skb_tstamp_tx" title="skb_tstamp_tx"><code class="xref c c-func docutils literal notranslate"><span class="pre">skb_tstamp_tx()</span></code></a> with the original skb, the raw
hardware time stamp. <a class="reference internal" href="kapi.html#c.skb_tstamp_tx" title="skb_tstamp_tx"><code class="xref c c-func docutils literal notranslate"><span class="pre">skb_tstamp_tx()</span></code></a> clones the original skb and
adds the timestamps, therefore the original skb has to be freed now.
If obtaining the hardware time stamp somehow fails, then the driver
should not fall back to software time stamping. The rationale is that
this would occur at a later time in the processing pipeline than other
software time stamping and therefore could lead to unexpected deltas
between time stamps.</p></li>
</ul>
</section>
<section id="special-considerations-for-stacked-ptp-hardware-clocks">
<h3>3.2 Special considerations for stacked PTP Hardware Clocks<a class="headerlink" href="#special-considerations-for-stacked-ptp-hardware-clocks" title="Link to this heading">¶</a></h3>
<p>There are situations when there may be more than one PHC (PTP Hardware Clock)
in the data path of a packet. The kernel has no explicit mechanism to allow the
user to select which PHC to use for timestamping Ethernet frames. Instead, the
assumption is that the outermost PHC is always the most preferable, and that
kernel drivers collaborate towards achieving that goal. Currently there are 3
cases of stacked PHCs, detailed below:</p>
<section id="dsa-distributed-switch-architecture-switches">
<h4>3.2.1 DSA (Distributed Switch Architecture) switches<a class="headerlink" href="#dsa-distributed-switch-architecture-switches" title="Link to this heading">¶</a></h4>
<p>These are Ethernet switches which have one of their ports connected to an
(otherwise completely unaware) host Ethernet interface, and perform the role of
a port multiplier with optional forwarding acceleration features. Each DSA
switch port is visible to the user as a standalone (virtual) network interface,
and its network I/O is performed, under the hood, indirectly through the host
interface (redirecting to the host port on TX, and intercepting frames on RX).</p>
<p>When a DSA switch is attached to a host port, PTP synchronization has to
suffer, since the switch’s variable queuing delay introduces a path delay
jitter between the host port and its PTP partner. For this reason, some DSA
switches include a timestamping clock of their own, and have the ability to
perform network timestamping on their own MAC, such that path delays only
measure wire and PHY propagation latencies. Timestamping DSA switches are
supported in Linux and expose the same ABI as any other network interface (save
for the fact that the DSA interfaces are in fact virtual in terms of network
I/O, they do have their own PHC). It is typical, but not mandatory, for all
interfaces of a DSA switch to share the same PHC.</p>
<p>By design, PTP timestamping with a DSA switch does not need any special
handling in the driver for the host port it is attached to. However, when the
host port also supports PTP timestamping, DSA will take care of intercepting
the <code class="docutils literal notranslate"><span class="pre">.ndo_eth_ioctl</span></code> calls towards the host port, and block attempts to enable
hardware timestamping on it. This is because the SO_TIMESTAMPING API does not
allow the delivery of multiple hardware timestamps for the same packet, so
anybody else except for the DSA switch port must be prevented from doing so.</p>
<p>In the generic layer, DSA provides the following infrastructure for PTP
timestamping:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">.port_txtstamp()</span></code>: a hook called prior to the transmission of
packets with a hardware TX timestamping request from user space.
This is required for two-step timestamping, since the hardware
timestamp becomes available after the actual MAC transmission, so the
driver must be prepared to correlate the timestamp with the original
packet so that it can re-enqueue the packet back into the socket’s
error queue. To save the packet for when the timestamp becomes
available, the driver can call <code class="docutils literal notranslate"><span class="pre">skb_clone_sk</span></code> , save the clone pointer
in skb->cb and enqueue a tx skb queue. Typically, a switch will have a
PTP TX timestamp register (or sometimes a FIFO) where the timestamp
becomes available. In case of a FIFO, the hardware might store
key-value pairs of PTP sequence ID/message type/domain number and the
actual timestamp. To perform the correlation correctly between the
packets in a queue waiting for timestamping and the actual timestamps,
drivers can use a BPF classifier (<code class="docutils literal notranslate"><span class="pre">ptp_classify_raw</span></code>) to identify
the PTP transport type, and <code class="docutils literal notranslate"><span class="pre">ptp_parse_header</span></code> to interpret the PTP
header fields. There may be an IRQ that is raised upon this
timestamp’s availability, or the driver might have to poll after
invoking <code class="docutils literal notranslate"><span class="pre">dev_queue_xmit()</span></code> towards the host interface.
One-step TX timestamping do not require packet cloning, since there is
no follow-up message required by the PTP protocol (because the
TX timestamp is embedded into the packet by the MAC), and therefore
user space does not expect the packet annotated with the TX timestamp
to be re-enqueued into its socket’s error queue.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">.port_rxtstamp()</span></code>: On RX, the BPF classifier is run by DSA to
identify PTP event messages (any other packets, including PTP general
messages, are not timestamped). The original (and only) timestampable
skb is provided to the driver, for it to annotate it with a timestamp,
if that is immediately available, or defer to later. On reception,
timestamps might either be available in-band (through metadata in the
DSA header, or attached in other ways to the packet), or out-of-band
(through another RX timestamping FIFO). Deferral on RX is typically
necessary when retrieving the timestamp needs a sleepable context. In
that case, it is the responsibility of the DSA driver to call
<code class="docutils literal notranslate"><span class="pre">netif_rx()</span></code> on the freshly timestamped skb.</p></li>
</ul>
</section>
<section id="ethernet-phys">
<h4>3.2.2 Ethernet PHYs<a class="headerlink" href="#ethernet-phys" title="Link to this heading">¶</a></h4>
<p>These are devices that typically fulfill a Layer 1 role in the network stack,
hence they do not have a representation in terms of a network interface as DSA
switches do. However, PHYs may be able to detect and timestamp PTP packets, for
performance reasons: timestamps taken as close as possible to the wire have the
potential to yield a more stable and precise synchronization.</p>
<p>A PHY driver that supports PTP timestamping must create a <code class="docutils literal notranslate"><span class="pre">struct</span>
<span class="pre">mii_timestamper</span></code> and add a pointer to it in <code class="docutils literal notranslate"><span class="pre">phydev->mii_ts</span></code>. The presence
of this pointer will be checked by the networking stack.</p>
<p>Since PHYs do not have network interface representations, the timestamping and
ethtool ioctl operations for them need to be mediated by their respective MAC
driver. Therefore, as opposed to DSA switches, modifications need to be done
to each individual MAC driver for PHY timestamping support. This entails:</p>
<ul>
<li><p>Checking, in <code class="docutils literal notranslate"><span class="pre">.ndo_eth_ioctl</span></code>, whether <code class="docutils literal notranslate"><span class="pre">phy_has_hwtstamp(netdev->phydev)</span></code>
is true or not. If it is, then the MAC driver should not process this request
but instead pass it on to the PHY using <code class="docutils literal notranslate"><span class="pre">phy_mii_ioctl()</span></code>.</p></li>
<li><p>On RX, special intervention may or may not be needed, depending on the
function used to deliver skb’s up the network stack. In the case of plain
<code class="docutils literal notranslate"><span class="pre">netif_rx()</span></code> and similar, MAC drivers must check whether
<code class="docutils literal notranslate"><span class="pre">skb_defer_rx_timestamp(skb)</span></code> is necessary or not - and if it is, don’t
call <code class="docutils literal notranslate"><span class="pre">netif_rx()</span></code> at all. If <code class="docutils literal notranslate"><span class="pre">CONFIG_NETWORK_PHY_TIMESTAMPING</span></code> is
enabled, and <code class="docutils literal notranslate"><span class="pre">skb->dev->phydev->mii_ts</span></code> exists, its <code class="docutils literal notranslate"><span class="pre">.rxtstamp()</span></code> hook
will be called now, to determine, using logic very similar to DSA, whether
deferral for RX timestamping is necessary. Again like DSA, it becomes the
responsibility of the PHY driver to send the packet up the stack when the
timestamp is available.</p>
<p>For other skb receive functions, such as <code class="docutils literal notranslate"><span class="pre">napi_gro_receive</span></code> and
<code class="docutils literal notranslate"><span class="pre">netif_receive_skb</span></code>, the stack automatically checks whether
<code class="docutils literal notranslate"><span class="pre">skb_defer_rx_timestamp()</span></code> is necessary, so this check is not needed inside
the driver.</p>
</li>
<li><p>On TX, again, special intervention might or might not be needed. The
function that calls the <code class="docutils literal notranslate"><span class="pre">mii_ts->txtstamp()</span></code> hook is named
<code class="docutils literal notranslate"><span class="pre">skb_clone_tx_timestamp()</span></code>. This function can either be called directly
(case in which explicit MAC driver support is indeed needed), but the
function also piggybacks from the <code class="docutils literal notranslate"><span class="pre">skb_tx_timestamp()</span></code> call, which many MAC
drivers already perform for software timestamping purposes. Therefore, if a
MAC supports software timestamping, it does not need to do anything further
at this stage.</p></li>
</ul>
</section>
<section id="mii-bus-snooping-devices">
<h4>3.2.3 MII bus snooping devices<a class="headerlink" href="#mii-bus-snooping-devices" title="Link to this heading">¶</a></h4>
<p>These perform the same role as timestamping Ethernet PHYs, save for the fact
that they are discrete devices and can therefore be used in conjunction with
any PHY even if it doesn’t support timestamping. In Linux, they are
discoverable and attachable to a <code class="docutils literal notranslate"><span class="pre">struct</span> <span class="pre">phy_device</span></code> through Device Tree, and
for the rest, they use the same mii_ts infrastructure as those. See
Documentation/devicetree/bindings/ptp/timestamper.txt for more details.</p>
</section>
<section id="other-caveats-for-mac-drivers">
<h4>3.2.4 Other caveats for MAC drivers<a class="headerlink" href="#other-caveats-for-mac-drivers" title="Link to this heading">¶</a></h4>
<p>The use of stacked PHCs may uncover MAC driver bugs which were impossible to
trigger without them. One example has to do with this line of code, already
presented earlier:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
</pre></div>
</div>
<p>Any TX timestamping logic, be it a plain MAC driver, a DSA switch driver, a PHY
driver or a MII bus snooping device driver, should set this flag.
But a MAC driver that is unaware of PHC stacking might get tripped up by
somebody other than itself setting this flag, and deliver a duplicate
timestamp.
For example, a typical driver design for TX timestamping might be to split the
transmission part into 2 portions:</p>
<ol class="arabic simple">
<li><p>“TX”: checks whether PTP timestamping has been previously enabled through
the <code class="docutils literal notranslate"><span class="pre">.ndo_eth_ioctl</span></code> (”<code class="docutils literal notranslate"><span class="pre">priv->hwtstamp_tx_enabled</span> <span class="pre">==</span> <span class="pre">true</span></code>”) and the
current skb requires a TX timestamp (”<code class="docutils literal notranslate"><span class="pre">skb_shinfo(skb)->tx_flags</span> <span class="pre">&</span>
<span class="pre">SKBTX_HW_TSTAMP</span></code>”). If this is true, it sets the
“<code class="docutils literal notranslate"><span class="pre">skb_shinfo(skb)->tx_flags</span> <span class="pre">|=</span> <span class="pre">SKBTX_IN_PROGRESS</span></code>” flag. Note: as
described above, in the case of a stacked PHC system, this condition should
never trigger, as this MAC is certainly not the outermost PHC. But this is
not where the typical issue is. Transmission proceeds with this packet.</p></li>
<li><p>“TX confirmation”: Transmission has finished. The driver checks whether it
is necessary to collect any TX timestamp for it. Here is where the typical
issues are: the MAC driver takes a shortcut and only checks whether
“<code class="docutils literal notranslate"><span class="pre">skb_shinfo(skb)->tx_flags</span> <span class="pre">&</span> <span class="pre">SKBTX_IN_PROGRESS</span></code>” was set. With a stacked
PHC system, this is incorrect because this MAC driver is not the only entity
in the TX data path who could have enabled SKBTX_IN_PROGRESS in the first
place.</p></li>
</ol>
<p>The correct solution for this problem is for MAC drivers to have a compound
check in their “TX confirmation” portion, not only for
“<code class="docutils literal notranslate"><span class="pre">skb_shinfo(skb)->tx_flags</span> <span class="pre">&</span> <span class="pre">SKBTX_IN_PROGRESS</span></code>”, but also for
“<code class="docutils literal notranslate"><span class="pre">priv->hwtstamp_tx_enabled</span> <span class="pre">==</span> <span class="pre">true</span></code>”. Because the rest of the system ensures
that PTP timestamping is not enabled for anything other than the outermost PHC,
this enhanced check will avoid delivering a duplicated TX timestamp to user
space.</p>
</section>
</section>
</section>
</section>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©The kernel development community.
|
Powered by <a href="https://www.sphinx-doc.org/">Sphinx 8.1.3</a>
& <a href="https://alabaster.readthedocs.io">Alabaster 0.7.16</a>
|
<a href="../_sources/networking/timestamping.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>