trace: add ability to do simple printf logging via systemtap

The dtrace systemtap trace backend for QEMU is very powerful but it is
also somewhat unfriendly to users who aren't familiar with systemtap,
or who don't need its power right now.

  stap -e "....some strange script...."

The 'log' backend for QEMU by comparison is very crude but incredibly
easy to use:

 $ qemu -d trace:qio* ...some args...
 23266@1547735759.137292:qio_channel_socket_new Socket new ioc=0x563a8a39d400
 23266@1547735759.137305:qio_task_new Task new task=0x563a891d0570 source=0x563a8a39d400 func=0x563a86f1e6c0 opaque=0x563a89078000
 23266@1547735759.137326:qio_task_thread_start Task thread start task=0x563a891d0570 worker=0x563a86f1ce50 opaque=0x563a891d9d90
 23273@1547735759.137491:qio_task_thread_run Task thread run task=0x563a891d0570
 23273@1547735759.137503:qio_channel_socket_connect_sync Socket connect sync ioc=0x563a8a39d400 addr=0x563a891d9d90
 23273@1547735759.138108:qio_channel_socket_connect_fail Socket connect fail ioc=0x563a8a39d400

This commit introduces a way to do simple printf style logging of probe
points using systemtap. In particular it creates another set of tapsets,
one per emulator:

  /usr/share/systemtap/tapset/qemu-*-log.stp

These pre-define probe functions which simply call printf() on their
arguments. The printf() format string is taken from the normal
trace-events files, with a little munging to the format specifiers
to cope with systemtap's more restrictive syntax.

With this you can now do

 $ stap -e 'probe qemu.system.x86_64.log.qio*{}'
 22806@1547735341399856820 qio_channel_socket_new Socket new ioc=0x56135d1d7c00
 22806@1547735341399862570 qio_task_new Task new task=0x56135cd66eb0 source=0x56135d1d7c00 func=0x56135af746c0 opaque=0x56135bf06400
 22806@1547735341399865943 qio_task_thread_start Task thread start task=0x56135cd66eb0 worker=0x56135af72e50 opaque=0x56135c071d70
 22806@1547735341399976816 qio_task_thread_run Task thread run task=0x56135cd66eb0

We go one step further though and introduce a 'qemu-trace-stap' tool to
make this even easier

 $ qemu-trace-stap run qemu-system-x86_64 'qio*'
 22806@1547735341399856820 qio_channel_socket_new Socket new ioc=0x56135d1d7c00
 22806@1547735341399862570 qio_task_new Task new task=0x56135cd66eb0 source=0x56135d1d7c00 func=0x56135af746c0 opaque=0x56135bf06400
 22806@1547735341399865943 qio_task_thread_start Task thread start task=0x56135cd66eb0 worker=0x56135af72e50 opaque=0x56135c071d70
 22806@1547735341399976816 qio_task_thread_run Task thread run task=0x56135cd66eb0

This tool is clever in that it will automatically change the
SYSTEMTAP_TAPSET env variable to point to the directory containing the
right set of probes for the QEMU binary path you give it. This is useful
if you have QEMU installed in /usr but are trying to test and trace a
binary in /home/berrange/usr/qemu-git. In that case you'd do

 $ qemu-trace-stap run /home/berrange/usr/qemu-git/bin/qemu-system-x86_64 'qio*'

And it'll make sure /home/berrange/usr/qemu-git/share/systemtap/tapset
is used for the trace session

The 'qemu-trace-stap' script takes a verbose arg so you can understand
what it is running

 $ qemu-trace-stap run /home/berrange/usr/qemu-git/bin/qemu-system-x86_64 'qio*'
 Using tapset dir '/home/berrange/usr/qemu-git/share/systemtap/tapset' for binary '/home/berrange/usr/qemu-git/bin/qemu-system-x86_64'
 Compiling script 'probe qemu.system.x86_64.log.qio* {}'
 Running script, <Ctrl>-c to quit
 ...trace output...

It can enable multiple probes at once

 $ qemu-trace-stap run qemu-system-x86_64 'qio*' 'qcrypto*' 'buffer*'

By default it monitors all existing running processes and all future
launched proceses. This can be restricted to a specific PID using the
--pid arg

 $ qemu-trace-stap run --pid 2532 qemu-system-x86_64 'qio*'

Finally if you can't remember what probes are valid it can tell you

 $ qemu-trace-stap list qemu-system-x86_64
 ahci_check_irq
 ahci_cmd_done
 ahci_dma_prepare_buf
 ahci_dma_prepare_buf_fail
 ahci_dma_rw_buf
 ahci_irq_lower
 ...snip...

Or list just those matching a prefix pattern

 $ qemu-trace-stap list -v qemu-system-x86_64 'qio*'
 Using tapset dir '/home/berrange/usr/qemu-git/share/systemtap/tapset' for binary '/home/berrange/usr/qemu-git/bin/qemu-system-x86_64'
 Listing probes with name 'qemu.system.x86_64.log.qio*'
 qio_channel_command_abort
 qio_channel_command_new_pid
 qio_channel_command_new_spawn
 qio_channel_command_wait
 qio_channel_file_new_fd
 ...snip...

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20190123120016.4538-5-berrange@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

1 files changed, 140 insertions, 0 deletions

diff --git a/scripts/qemu-trace-stap.texi b/scripts/qemu-trace-stap.texi
new file mode 100644
index 0000000..07bb9eb
--- /dev/null
+++ b/scripts/qemu-trace-stap.texi
@@ -0,0 +1,140 @@
+@example
+@c man begin SYNOPSIS
+@command{qemu-trace-stap} @var{GLOBAL-OPTIONS} @var{COMMAND} @var{COMMAND-OPTIONS} @var{ARGS...}
+@c man end
+@end example
+
+@c man begin DESCRIPTION
+
+The @command{qemu-trace-stap} program facilitates tracing of the execution
+of QEMU emulators using SystemTap.
+
+It is required to have the SystemTap runtime environment installed to use
+this program, since it is a wrapper around execution of the @command{stap}
+program.
+
+@c man end
+
+@c man begin OPTIONS
+
+The following global options may be used regardless of which command
+is executed:
+
+@table @option
+@item @var{--verbose}, @var{-v}
+
+Display verbose information about command execution.
+
+@end table
+
+The following commands are valid:
+
+@table @option
+
+@item @var{list} @var{BINARY} @var{PATTERN...}
+
+List all the probe names provided by @var{BINARY} that match
+@var{PATTERN}.
+
+If @var{BINARY} is not an absolute path, it will be located by searching
+the directories listed in the @code{$PATH} environment variable.
+
+@var{PATTERN} is a plain string that is used to filter the results of
+this command. It may optionally contain a @code{*} wildcard to facilitate
+matching multiple probes without listing each one explicitly. Multiple
+@var{PATTERN} arguments may be given, causing listing of probes that match
+any of the listed names. If no @var{PATTERN} is given, the all possible
+probes will be listed.
+
+For example, to list all probes available in the @command{qemu-system-x86_64}
+binary:
+
+@example
+$ qemu-trace-stap list qemu-system-x86_64
+@end example
+
+To filter the list to only cover probes related to QEMU's cryptographic
+subsystem, in a binary outside @code{$PATH}
+
+@example
+$ qemu-trace-stap list /opt/qemu/4.0.0/bin/qemu-system-x86_64 'qcrypto*'
+@end example
+
+
+@item @var{run} @var{OPTIONS} @var{BINARY} @var{PATTERN...}
+
+Run a trace session, printing formatted output any time a process that is
+executing @var{BINARY} triggers a probe matching @var{PATTERN}.
+
+If @var{BINARY} is not an absolute path, it will be located by searching
+the directories listed in the @code{$PATH} environment variable.
+
+@var{PATTERN} is a plain string that matches a probe name shown by the
+@var{list} command. It may optionally contain a @code{*} wildcard to
+facilitate matching multiple probes without listing each one explicitly.
+Multiple @var{PATTERN} arguments may be given, causing all matching probes
+to be monitored. At least one @var{PATTERN} is required, since stap is not
+capable of tracing all known QEMU probes concurrently without overflowing
+its trace buffer.
+
+Invocation of this command does not need to be synchronized with
+invocation of the QEMU process(es). It will match probes on all
+existing running processes and all future launched processes,
+unless told to only monitor a specific process.
+
+Valid command specific options are:
+
+@table @option
+@item @var{--pid=PID}, @var{-p PID}
+
+Restrict the tracing session so that it only triggers for the process
+identified by @code{PID}.
+
+@end table
+
+For example, to monitor all processes executing @command{qemu-system-x86_64}
+as found on $PATH, displaying all I/O related probes:
+
+@example
+$ qemu-trace-stap run qemu-system-x86_64 'qio*'
+@end example
+
+To monitor only the QEMU process with PID 1732
+
+@example
+$ qemu-trace-stap run --pid=1732 qemu-system-x86_64 'qio*'
+@end example
+
+To monitor QEMU processes running an alternative binary outside of
+@code{$PATH}, displaying verbose information about setup of the
+tracing environment:
+
+@example
+$ qemu-trace-stap -v run /opt/qemu/4.0.0/qemu-system-x86_64 'qio*'
+@end example
+
+@end table
+
+@c man end
+
+@ignore
+
+@setfilename qemu-trace-stap
+@settitle QEMU SystemTap trace tool
+
+@c man begin LICENSE
+
+Copyright (C) 2019 Red Hat, Inc.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+
+@c man end
+
+@c man begin SEEALSO
+qemu(1), stap(1)
+@c man end
+
+@end ignore