<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Chapter 4. Useful SystemTap Scripts</title><link rel="stylesheet" href="./Common_Content/css/default.css" type="text/css" /><meta name="generator" content="publican 1.6" /><meta name="package" content="Systemtap-SystemTap_Beginners_Guide-1.0-en-US-2.0-2" /><link rel="home" href="index.html" title="SystemTap Beginners Guide" /><link rel="up" href="index.html" title="SystemTap Beginners Guide" /><link rel="prev" href="understanding-tapsets.html" title="3.6. Tapsets" /><link rel="next" href="mainsect-disk.html" title="4.2. Disk" /></head><body class=""><p id="title"><a class="left" href="http://www.fedoraproject.org"><img src="Common_Content/images/image_left.png" alt="Product Site" /></a><a class="right" href="http://docs.fedoraproject.org"><img src="Common_Content/images/image_right.png" alt="Documentation Site" /></a></p><ul class="docnav"><li class="previous"><a accesskey="p" href="understanding-tapsets.html"><strong>Prev</strong></a></li><li class="next"><a accesskey="n" href="mainsect-disk.html"><strong>Next</strong></a></li></ul><div xml:lang="en-US" class="chapter" title="Chapter 4. Useful SystemTap Scripts" lang="en-US"><div class="titlepage"><div><div><h2 class="title" id="useful-systemtap-scripts">Chapter 4. Useful SystemTap Scripts</h2></div></div></div><div class="toc"><dl><dt><span class="section"><a href="useful-systemtap-scripts.html#mainsect-network">4.1. Network</a></span></dt><dd><dl><dt><span class="section"><a href="useful-systemtap-scripts.html#nettopsect">4.1.1. Network Profiling</a></span></dt><dt><span class="section"><a href="useful-systemtap-scripts.html#sockettracesect">4.1.2. Tracing Functions Called in Network Socket Code</a></span></dt><dt><span class="section"><a href="useful-systemtap-scripts.html#tcpconnectionssect">4.1.3. Monitoring Incoming TCP Connections</a></span></dt><dt><span class="section"><a href="useful-systemtap-scripts.html#tcpdumplikesect">4.1.4. Monitoring TCP Packets</a></span></dt><dt><span class="section"><a href="useful-systemtap-scripts.html#dropwatchsect">4.1.5. Monitoring Network Packets Drops in Kernel</a></span></dt></dl></dd><dt><span class="section"><a href="mainsect-disk.html">4.2. Disk</a></span></dt><dd><dl><dt><span class="section"><a href="mainsect-disk.html#disktop">4.2.1. Summarizing Disk Read/Write Traffic</a></span></dt><dt><span class="section"><a href="iotimesect.html">4.2.2. Tracking I/O Time For Each File Read or Write</a></span></dt><dt><span class="section"><a href="traceiosect.html">4.2.3. Track Cumulative IO</a></span></dt><dt><span class="section"><a href="traceio2sect.html">4.2.4. I/O Monitoring (By Device)</a></span></dt><dt><span class="section"><a href="inodewatchsect.html">4.2.5. Monitoring Reads and Writes to a File</a></span></dt><dt><span class="section"><a href="inodewatch2sect.html">4.2.6. Monitoring Changes to File Attributes</a></span></dt><dt><span class="section"><a href="ioblktimesect.html">4.2.7. Periodically Print I/O Block Time</a></span></dt></dl></dd><dt><span class="section"><a href="mainsect-profiling.html">4.3. Profiling</a></span></dt><dd><dl><dt><span class="section"><a href="mainsect-profiling.html#countcallssect">4.3.1. Counting Function Calls Made</a></span></dt><dt><span class="section"><a href="paracallgraph.html">4.3.2. Call Graph Tracing</a></span></dt><dt><span class="section"><a href="threadtimessect.html">4.3.3. Determining Time Spent in Kernel and User Space</a></span></dt><dt><span class="section"><a href="timeoutssect.html">4.3.4. Monitoring Polling Applications</a></span></dt><dt><span class="section"><a href="topsyssect.html">4.3.5. Tracking Most Frequently Used System Calls</a></span></dt><dt><span class="section"><a href="syscallsbyprocpidsect.html">4.3.6. Tracking System Call Volume Per Process</a></span></dt></dl></dd><dt><span class="section"><a href="futexcontentionsect.html">4.4. Identifying Contended User-Space Locks</a></span></dt></dl></div><a id="id2939982" class="indexterm"></a><a id="id2939972" class="indexterm"></a><a id="id2939967" class="indexterm"></a><a id="id2939958" class="indexterm"></a><div class="para">
This chapter enumerates several SystemTap scripts you can use to monitor and investigate different subsystems. All of these scripts are available at <code class="filename">/usr/share/systemtap/testsuite/systemtap.examples/</code> once you install the <code class="filename">systemtap-testsuite</code> RPM.
</div><div class="section" title="4.1. Network"><div class="titlepage"><div><div><h2 class="title" id="mainsect-network">4.1. Network</h2></div></div></div><div class="para">
The following sections showcase scripts that trace network-related functions and build a profile of network activity.
</div><div xml:lang="en-US" class="section" title="4.1.1. Network Profiling" lang="en-US"><div class="titlepage"><div><div><h3 class="title" id="nettopsect">4.1.1. Network Profiling</h3></div></div></div><a id="id2939912" class="indexterm"></a><a id="id2939900" class="indexterm"></a><a id="id2939887" class="indexterm"></a><a id="id2939866" class="indexterm"></a><a id="id2724225" class="indexterm"></a><div class="para">
This section describes how to profile network activity. <a class="xref" href="useful-systemtap-scripts.html#nettop" title="nettop.stp">nettop.stp</a> provides a glimpse into how much network traffic each process is generating on a machine.
</div><div class="formalpara"><h5 class="formalpara" id="nettop">nettop.stp</h5>
<pre class="programlisting">
#! /usr/bin/env stap
global ifxmit, ifrecv
global ifmerged
probe netdev.transmit
{
ifxmit[pid(), dev_name, execname(), uid()] <<< length
}
probe netdev.receive
{
ifrecv[pid(), dev_name, execname(), uid()] <<< length
}
function print_activity()
{
printf("%5s %5s %-7s %7s %7s %7s %7s %-15s\n",
"PID", "UID", "DEV", "XMIT_PK", "RECV_PK",
"XMIT_KB", "RECV_KB", "COMMAND")
foreach ([pid, dev, exec, uid] in ifrecv) {
ifmerged[pid, dev, exec, uid] += @count(ifrecv[pid,dev,exec,uid]);
}
foreach ([pid, dev, exec, uid] in ifxmit) {
ifmerged[pid, dev, exec, uid] += @count(ifxmit[pid,dev,exec,uid]);
}
foreach ([pid, dev, exec, uid] in ifmerged-) {
n_xmit = @count(ifxmit[pid, dev, exec, uid])
n_recv = @count(ifrecv[pid, dev, exec, uid])
printf("%5d %5d %-7s %7d %7d %7d %7d %-15s\n",
pid, uid, dev, n_xmit, n_recv,
n_xmit ? @sum(ifxmit[pid, dev, exec, uid])/1024 : 0,
n_recv ? @sum(ifrecv[pid, dev, exec, uid])/1024 : 0,
exec)
}
print("\n")
delete ifxmit
delete ifrecv
delete ifmerged
}
probe timer.ms(5000), end, error
{
print_activity()
}
</pre>
</div><a id="id2953137" class="indexterm"></a><a id="id2715803" class="indexterm"></a><a id="id2924904" class="indexterm"></a><div class="para">
Note that <code class="command">function print_activity()</code> uses the following expressions:
</div><pre class="screen">
n_xmit ? @sum(ifxmit[pid, dev, exec, uid])/1024 : 0
n_recv ? @sum(ifrecv[pid, dev, exec, uid])/1024 : 0
</pre><div class="para">
These expressions are if/else conditionals. The first statement is simply a more concise way of writing the following psuedo code:
</div><pre class="screen">
if n_recv != 0 then
@sum(ifrecv[pid, dev, exec, uid])/1024
else
0
</pre><div class="para">
<a class="xref" href="useful-systemtap-scripts.html#nettop" title="nettop.stp">nettop.stp</a> tracks which processes are generating network traffic on the system, and provides the following information about each process:
</div><div class="itemizedlist"><ul><li class="listitem"><div class="para">
<code class="computeroutput">PID</code> — the ID of the listed process.
</div></li><li class="listitem"><div class="para">
<code class="computeroutput">UID</code> — user ID. A user ID of <code class="computeroutput">0</code> refers to the root user.
</div></li><li class="listitem"><div class="para">
<code class="computeroutput">DEV</code> — which ethernet device the process used to send / receive data (e.g. eth0, eth1)
</div></li><li class="listitem"><div class="para">
<code class="computeroutput">XMIT_PK</code> — number of packets transmitted by the process
</div></li><li class="listitem"><div class="para">
<code class="computeroutput">RECV_PK</code> — number of packets received by the process
</div></li><li class="listitem"><div class="para">
<code class="computeroutput">XMIT_KB</code> — amount of data sent by the process, in kilobytes
</div></li><li class="listitem"><div class="para">
<code class="computeroutput">RECV_KB</code> — amount of data received by the service, in kilobytes
</div></li></ul></div><div class="para">
<a class="xref" href="useful-systemtap-scripts.html#nettop" title="nettop.stp">nettop.stp</a> provides network profile sampling every 5 seconds. You can change this setting by editing <code class="command">probe timer.ms(5000)</code> accordingly. <a class="xref" href="useful-systemtap-scripts.html#nettopoutput" title="Example 4.1. nettop.stp Sample Output">Example 4.1, “nettop.stp Sample Output”</a> contains an excerpt of the output from <a class="xref" href="useful-systemtap-scripts.html#nettop" title="nettop.stp">nettop.stp</a> over a 20-second period:
</div><div class="example" id="nettopoutput"><div class="example-contents"><pre class="screen">
[...]
PID UID DEV XMIT_PK RECV_PK XMIT_KB RECV_KB COMMAND
0 0 eth0 0 5 0 0 swapper
11178 0 eth0 2 0 0 0 synergyc
PID UID DEV XMIT_PK RECV_PK XMIT_KB RECV_KB COMMAND
2886 4 eth0 79 0 5 0 cups-polld
11362 0 eth0 0 61 0 5 firefox
0 0 eth0 3 32 0 3 swapper
2886 4 lo 4 4 0 0 cups-polld
11178 0 eth0 3 0 0 0 synergyc
PID UID DEV XMIT_PK RECV_PK XMIT_KB RECV_KB COMMAND
0 0 eth0 0 6 0 0 swapper
2886 4 lo 2 2 0 0 cups-polld
11178 0 eth0 3 0 0 0 synergyc
3611 0 eth0 0 1 0 0 Xorg
PID UID DEV XMIT_PK RECV_PK XMIT_KB RECV_KB COMMAND
0 0 eth0 3 42 0 2 swapper
11178 0 eth0 43 1 3 0 synergyc
11362 0 eth0 0 7 0 0 firefox
3897 0 eth0 0 1 0 0 multiload-apple
[...]
</pre></div><h6>Example 4.1. <a class="xref" href="useful-systemtap-scripts.html#nettop" title="nettop.stp">nettop.stp</a> Sample Output</h6></div><br class="example-break" /></div><div xml:lang="en-US" class="section" title="4.1.2. Tracing Functions Called in Network Socket Code" lang="en-US"><div class="titlepage"><div><div><h3 class="title" id="sockettracesect">4.1.2. Tracing Functions Called in Network Socket Code</h3></div></div></div><a id="id2743916" class="indexterm"></a><a id="id3099800" class="indexterm"></a><a id="id2755656" class="indexterm"></a><a id="id3092623" class="indexterm"></a><a id="id2956905" class="indexterm"></a><a id="id2954592" class="indexterm"></a><a id="id2925008" class="indexterm"></a><a id="id2717585" class="indexterm"></a><div class="para">
This section describes how to trace functions called from the kernel's <code class="filename">net/socket.c</code> file. This task helps you identify, in finer detail, how each process interacts with the network at the kernel level.
</div><div class="formalpara"><h5 class="formalpara" id="sockettrace">socket-trace.stp</h5>
<pre class="programlisting">
#! /usr/bin/env stap
probe kernel.function("*@net/socket.c").call {
printf ("%s -> %s\n", thread_indent(1), probefunc())
}
probe kernel.function("*@net/socket.c").return {
printf ("%s <- %s\n", thread_indent(-1), probefunc())
}
</pre>
</div><div class="para">
<a class="xref" href="useful-systemtap-scripts.html#sockettrace" title="socket-trace.stp">socket-trace.stp</a> is identical to <a class="xref" href="systemtapscript-handler.html#thread_indent" title="Example 3.6. thread_indent.stp">Example 3.6, “thread_indent.stp”</a>, which was earlier used in <a class="xref" href="systemtapscript-handler.html#systemtapscript-functions" title="SystemTap Functions">SystemTap Functions</a> to illustrate how <code class="command">thread_indent()</code> works.
</div><div class="example" id="sockettraceoutput"><div class="example-contents"><pre class="screen">
[...]
0 Xorg(3611): -> sock_poll
3 Xorg(3611): <- sock_poll
0 Xorg(3611): -> sock_poll
3 Xorg(3611): <- sock_poll
0 gnome-terminal(11106): -> sock_poll
5 gnome-terminal(11106): <- sock_poll
0 scim-bridge(3883): -> sock_poll
3 scim-bridge(3883): <- sock_poll
0 scim-bridge(3883): -> sys_socketcall
4 scim-bridge(3883): -> sys_recv
8 scim-bridge(3883): -> sys_recvfrom
12 scim-bridge(3883):-> sock_from_file
16 scim-bridge(3883):<- sock_from_file
20 scim-bridge(3883):-> sock_recvmsg
24 scim-bridge(3883):<- sock_recvmsg
28 scim-bridge(3883): <- sys_recvfrom
31 scim-bridge(3883): <- sys_recv
35 scim-bridge(3883): <- sys_socketcall
[...]
</pre></div><h6>Example 4.2. <a class="xref" href="useful-systemtap-scripts.html#sockettrace" title="socket-trace.stp">socket-trace.stp</a> Sample Output</h6></div><br class="example-break" /><div class="para">
<a class="xref" href="useful-systemtap-scripts.html#sockettraceoutput" title="Example 4.2. socket-trace.stp Sample Output">Example 4.2, “socket-trace.stp Sample Output”</a> contains a 3-second excerpt of the output for <a class="xref" href="useful-systemtap-scripts.html#sockettrace" title="socket-trace.stp">socket-trace.stp</a>. For more information about the output of this script as provided by <code class="command">thread_indent()</code>, refer to <a class="xref" href="systemtapscript-handler.html#systemtapscript-functions" title="SystemTap Functions">SystemTap Functions</a> <a class="xref" href="systemtapscript-handler.html#thread_indent" title="Example 3.6. thread_indent.stp">Example 3.6, “thread_indent.stp”</a>.
</div></div><div xml:lang="en-US" class="section" title="4.1.3. Monitoring Incoming TCP Connections" lang="en-US"><div class="titlepage"><div><div><h3 class="title" id="tcpconnectionssect">4.1.3. Monitoring Incoming TCP Connections</h3></div></div></div><a id="id2726912" class="indexterm"></a><a id="id2721422" class="indexterm"></a><a id="id4354322" class="indexterm"></a><a id="id2989069" class="indexterm"></a><a id="id2750920" class="indexterm"></a><div class="para">
This section illustrates how to monitor incoming TCP connections. This task is useful in identifying any unauthorized, suspicious, or otherwise unwanted network access requests in real time.
</div><div class="formalpara"><h5 class="formalpara" id="tcpconnections">tcp_connections.stp</h5>
<pre class="programlisting">
#! /usr/bin/env stap
probe begin {
printf("%6s %16s %6s %6s %16s\n",
"UID", "CMD", "PID", "PORT", "IP_SOURCE")
}
probe kernel.function("tcp_accept").return?,
kernel.function("inet_csk_accept").return? {
sock = $return
if (sock != 0)
printf("%6d %16s %6d %6d %16s\n", uid(), execname(), pid(),
inet_get_local_port(sock), inet_get_ip_source(sock))
}
</pre>
</div><div class="para">
While <a class="xref" href="useful-systemtap-scripts.html#tcpconnections" title="tcp_connections.stp">tcp_connections.stp</a> is running, it will print out the following information about any incoming TCP connections accepted by the system in real time:
</div><div class="itemizedlist"><ul><li class="listitem"><div class="para">
Current <code class="command">UID</code>
</div></li><li class="listitem"><div class="para">
<code class="command">CMD</code> - the command accepting the connection
</div></li><li class="listitem"><div class="para">
<code class="command">PID</code> of the command
</div></li><li class="listitem"><div class="para">
Port used by the connection
</div></li><li class="listitem"><div class="para">
IP address from which the TCP connection originated
</div></li></ul></div><div class="example" id="tcpconnectionsoutput"><div class="example-contents"><pre class="screen">
UID CMD PID PORT IP_SOURCE
0 sshd 3165 22 10.64.0.227
0 sshd 3165 22 10.64.0.227
</pre></div><h6>Example 4.3. <a class="xref" href="useful-systemtap-scripts.html#tcpconnections" title="tcp_connections.stp">tcp_connections.stp</a> Sample Output</h6></div><br class="example-break" /></div><div xml:lang="en-US" class="section" title="4.1.4. Monitoring TCP Packets" lang="en-US"><div class="titlepage"><div><div><h3 class="title" id="tcpdumplikesect">4.1.4. Monitoring TCP Packets</h3></div></div></div><a id="id2917156" class="indexterm"></a><a id="id2989512" class="indexterm"></a><a id="id3001994" class="indexterm"></a><a id="id2872284" class="indexterm"></a><a id="id2754139" class="indexterm"></a><div class="para">
This section illustrates how to monitor TCP packets received by the system. This is useful in analyzing network traffic generated by applications running on the system.
</div><div class="formalpara"><h5 class="formalpara" id="tcpdumplike">tcpdumplike.stp</h5>
<pre class="programlisting">
#! /usr/bin/env stap
// A TCP dump like example
probe begin, timer.s(1) {
printf("-----------------------------------------------------------------\n")
printf(" Source IP Dest IP SPort DPort U A P R S F \n")
printf("-----------------------------------------------------------------\n")
}
probe tcp.receive {
printf(" %15s %15s %5d %5d %d %d %d %d %d %d\n",
saddr, daddr, sport, dport, urg, ack, psh, rst, syn, fin)
}
</pre>
</div><div class="para">
While <a class="xref" href="useful-systemtap-scripts.html#tcpdumplike" title="tcpdumplike.stp">tcpdumplike.stp</a> is running, it will print out the following information about any received TCP packets in real time:
</div><div class="itemizedlist"><ul><li class="listitem"><div class="para">
Source and destination IP address (<code class="command">saddr</code>, <code class="command">daddr</code>, respectively)
</div></li><li class="listitem"><div class="para">
Source and destination ports (<code class="command">sport</code>, <code class="command">dport</code>, respectively)
</div></li><li class="listitem"><div class="para">
Packet flags
</div></li></ul></div><div class="para">
To determine the flags used by the packet, <a class="xref" href="useful-systemtap-scripts.html#tcpdumplike" title="tcpdumplike.stp">tcpdumplike.stp</a> uses the following functions:
</div><div class="itemizedlist"><ul><li class="listitem"><div class="para">
<code class="command">urg</code> - urgent
</div></li><li class="listitem"><div class="para">
<code class="command">ack</code> - acknowledgement
</div></li><li class="listitem"><div class="para">
<code class="command">psh</code> - push
</div></li><li class="listitem"><div class="para">
<code class="command">rst</code> - reset
</div></li><li class="listitem"><div class="para">
<code class="command">syn</code> - synchronize
</div></li><li class="listitem"><div class="para">
<code class="command">fin</code> - finished
</div></li></ul></div><div class="para">
The aforementioned functions return <code class="command">1</code> or <code class="command">0</code> to specify whether the packet uses the corresponding flag.
</div><div class="example" id="tcpdumplikeoutput"><div class="example-contents"><pre class="screen">
-----------------------------------------------------------------
Source IP Dest IP SPort DPort U A P R S F
-----------------------------------------------------------------
209.85.229.147 10.0.2.15 80 20373 0 1 1 0 0 0
92.122.126.240 10.0.2.15 80 53214 0 1 0 0 1 0
92.122.126.240 10.0.2.15 80 53214 0 1 0 0 0 0
209.85.229.118 10.0.2.15 80 63433 0 1 0 0 1 0
209.85.229.118 10.0.2.15 80 63433 0 1 0 0 0 0
209.85.229.147 10.0.2.15 80 21141 0 1 1 0 0 0
209.85.229.147 10.0.2.15 80 21141 0 1 1 0 0 0
209.85.229.147 10.0.2.15 80 21141 0 1 1 0 0 0
209.85.229.147 10.0.2.15 80 21141 0 1 1 0 0 0
209.85.229.147 10.0.2.15 80 21141 0 1 1 0 0 0
209.85.229.118 10.0.2.15 80 63433 0 1 1 0 0 0
[...]
</pre></div><h6>Example 4.4. <a class="xref" href="useful-systemtap-scripts.html#tcpdumplike" title="tcpdumplike.stp">tcpdumplike.stp</a> Sample Output</h6></div><br class="example-break" /></div><div xml:lang="en-US" class="section" title="4.1.5. Monitoring Network Packets Drops in Kernel" lang="en-US"><div class="titlepage"><div><div><h3 class="title" id="dropwatchsect">4.1.5. Monitoring Network Packets Drops in Kernel</h3></div></div></div><a id="id2862185" class="indexterm"></a><a id="id2743832" class="indexterm"></a><a id="id2765416" class="indexterm"></a><a id="id3008920" class="indexterm"></a><a id="id3033348" class="indexterm"></a><div class="para">
<a id="id4386981" class="indexterm"></a>
The network stack in Linux can discard packets for various reasons. Some Linux kernels include a tracepoint, <code class="command">kernel.trace("kfree_skb")</code>, which easily tracks where packets are discarded. <a class="xref" href="useful-systemtap-scripts.html#dropwatch" title="dropwatch.stp">dropwatch.stp</a> uses <code class="command">kernel.trace("kfree_skb")</code> to trace packet discards; the script summarizes which locations discard packets every five-second interval.
</div><div class="formalpara"><h5 class="formalpara" id="dropwatch">dropwatch.stp</h5>
<pre class="programlisting">
#! /usr/bin/env stap
############################################################
# Dropwatch.stp
# Author: Neil Horman <nhorman@redhat.com>
# An example script to mimic the behavior of the dropwatch utility
# http://fedorahosted.org/dropwatch
############################################################
# Array to hold the list of drop points we find
global locations
# Note when we turn the monitor on and off
probe begin { printf("Monitoring for dropped packets\n") }
probe end { printf("Stopping dropped packet monitor\n") }
# increment a drop counter for every location we drop at
probe kernel.trace("kfree_skb") { locations[$location] <<< 1 }
# Every 5 seconds report our drop locations
probe timer.sec(5)
{
printf("\n")
foreach (l in locations-) {
printf("%d packets dropped at location %p\n",
@count(locations[l]), l)
}
delete locations
}
</pre>
</div><div class="para">
The <code class="command">kernel.trace("kfree_skb")</code> traces which places in the kernel drop network packets. The <code class="command">kernel.trace("kfree_skb")</code> has two arguments: a pointer to the buffer being freed (<code class="command">$skb</code>) and the location in kernel code the buffer is being freed (<code class="command">$location</code>).
</div><div class="para">
Running the dropwatch.stp script 15 seconds would result in output similar in <a class="xref" href="useful-systemtap-scripts.html#dropwatchoutput" title="Example 4.5. dropwatch.stp Sample Output">Example 4.5, “dropwatch.stp Sample Output”</a>. The output lists the number of misses for tracepoint address and the actual address.
</div><div class="example" id="dropwatchoutput"><div class="example-contents"><pre class="screen">
Monitoring for dropped packets
51 packets dropped at location 0xffffffff8024cd0f
2 packets dropped at location 0xffffffff8044b472
51 packets dropped at location 0xffffffff8024cd0f
1 packets dropped at location 0xffffffff8044b472
97 packets dropped at location 0xffffffff8024cd0f
1 packets dropped at location 0xffffffff8044b472
Stopping dropped packet monitor
</pre></div><h6>Example 4.5. <a class="xref" href="useful-systemtap-scripts.html#dropwatch" title="dropwatch.stp">dropwatch.stp</a> Sample Output</h6></div><br class="example-break" /><div class="para">
To make the location of packet drops more meaningful, refer to the <code class="filename">/boot/System.map-`uname -r`</code> file. This file lists the starting addresses for each function, allowing you to map the addresses in the output of <a class="xref" href="useful-systemtap-scripts.html#dropwatchoutput" title="Example 4.5. dropwatch.stp Sample Output">Example 4.5, “dropwatch.stp Sample Output”</a> to a specific function name. Given the following snippet of the <code class="filename">/boot/System.map-`uname -r`</code> file, the address 0xffffffff8024cd0f maps to the function <code class="command">unix_stream_recvmsg</code> and the address 0xffffffff8044b472 maps to the function <code class="command">arp_rcv</code>:
</div><pre class="screen">
[...]
ffffffff8024c5cd T unlock_new_inode
ffffffff8024c5da t unix_stream_sendmsg
ffffffff8024c920 t unix_stream_recvmsg
ffffffff8024cea1 t udp_v4_lookup_longway
[...]
ffffffff8044addc t arp_process
ffffffff8044b360 t arp_rcv
ffffffff8044b487 t parp_redo
ffffffff8044b48c t arp_solicit
[...]
</pre></div></div></div><ul class="docnav"><li class="previous"><a accesskey="p" href="understanding-tapsets.html"><strong>Prev</strong>3.6. Tapsets</a></li><li class="up"><a accesskey="u" href="#"><strong>Up</strong></a></li><li class="home"><a accesskey="h" href="index.html"><strong>Home</strong></a></li><li class="next"><a accesskey="n" href="mainsect-disk.html"><strong>Next</strong>4.2. Disk</a></li></ul></body></html>
