LTTNG-EXPORT-MAPS(1)

NAME
lttng-export-maps - Export the counter values of an LTTng recording session as SQL

SYNOPSIS

lttng [GENERAL OPTIONS] export-maps [--session=SESSION] [--format=sqlite]

DESCRIPTION

The lttng export-maps command exports the current counter values of all the map channels (see lttng-add-map-channel(1)) of:

With the --session=SESSION option
    The recording session named SESSION.

Without the --session option
    The current recording session (see lttng-concepts(7) to learn more about the current recording session).

See lttng-concepts(7) to learn more about recording sessions, map channels, and counters.

Unlike lttng-show-maps(1), which offers a fixed, human-readable presentation with built-in filtering and aggregation, lttng export-maps performs no filtering and no aggregation: it exports every counter of every map channel as a self-contained SQL script for SQLite. Slice, filter, and aggregate the data afterwards with the full power of SQLite.

The default and only output format as of LTTng 2.16.0 is sqlite (see the --format option): an SQL script for the SQLite engine which creates a few tables and a vmap convenience view, and then inserts every counter within a single transaction. Pipe the script into sqlite3 to query the data, for example using an in-memory database:

    $ { lttng export-maps; echo 'SELECT * FROM vmap;' } | sqlite3 :memory:

See the “SQL SCHEMA” section below for the schema details and the “EXAMPLES” section below for a few query examples.

Important

    The lttng export-maps command does NOT provide an atomic snapshot of the counter values: the connected session daemon (see lttng-sessiond(8)) reads the values one by one, and LTTng tracers may concurrently update any of them while the command runs.

    Therefore, lttng export-maps does NOT guarantee that its output reflects the state of the counters at a single instant.

    To obtain a consistent view of the counter values, LTTng recommends to either:

        • Stop the recording session with the lttng-stop(1) command before running lttng export-maps.

        • Disable the recording event rules (see lttng-disable-event(1)) and/or remove the triggers (see lttng-remove-trigger(1)) having an “increment map value” action which targets the map channel to inspect.

    Moreover, lttng export-maps doesn’t guarantee the order in which it writes its INSERT INTO statements; don’t rely on it.

SQL SCHEMA

With --format=sqlite (the default), the command writes the following schema, followed by one INSERT INTO statement per row, all wrapped in a single BEGIN/COMMIT transaction:

    CREATE TABLE channels (
      id   INTEGER PRIMARY KEY,
      name TEXT NOT NULL,
      type TEXT NOT NULL CHECK (type IN ('kernel', 'user')),
      UNIQUE (name, type)
    );
    CREATE TABLE groups (
      id          INTEGER PRIMARY KEY,
      channel_id  INTEGER NOT NULL REFERENCES channels(id),
      type        TEXT NOT NULL
                  CHECK (type IN ('kernel-global', 'user-per-user',
                                  'user-per-process', 'shared')),
      owner_id    INTEGER,
      owner_name  TEXT,
      value_type  TEXT NOT NULL
                  CHECK (value_type IN ('signed-int-32', 'signed-int-64')),
      CHECK ((owner_id IS NULL) =
             (type NOT IN ('user-per-user', 'user-per-process')))
    );
    CREATE TABLE keys (
      id   INTEGER PRIMARY KEY,
      name TEXT NOT NULL UNIQUE
    );
    CREATE TABLE entries (
      group_id     INTEGER NOT NULL REFERENCES groups(id),
      part_id      INTEGER,
      key_id       INTEGER NOT NULL REFERENCES keys(id),
      value        INTEGER NOT NULL,
      has_overflow INTEGER NOT NULL CHECK (has_overflow IN (0, 1))
    );
    CREATE VIEW vmap AS
    SELECT c.name AS channel_name,
           c.type AS channel_type,
           g.type AS group_type,
           g.owner_id,
           g.owner_name,
           g.value_type,
           e.part_id,
           k.name AS key,
           e.value,
           e.has_overflow
    FROM entries e
    JOIN groups g   ON e.group_id   = g.id
    JOIN channels c ON g.channel_id = c.id
    JOIN keys k     ON e.key_id     = k.id;

The numeric id columns are arbitrary surrogate keys which only serve to join the tables; don’t rely on their specific values nor on the row order of the INSERT INTO statements.

Most queries only need the vmap view, of which the columns are:

channel_name
    Map channel name.

channel_type
    Map channel type: kernel or user.

group_type
    Counter group type:

    kernel-global
        System-wide Linux kernel counters.

    user-per-user
        User space counters of a single Unix user; owner_id is the Unix user ID.

    user-per-process
        User space counters of a single process; owner_id is the process ID.

    shared
        User space map channel-wide shared counters (see lttng-concepts(7)).

owner_id
    Unix user ID (user-per-user group type) or process ID (user-per-process group type), or NULL for the kernel-global and shared group types.

owner_name
    Human-readable name of the owner: a Unix user name (user-per-user group type) or a process/command name (user-per-process group type).

    This column is informational only: LTTng makes no guarantee that the name is set (it may be NULL), that it’s unique amongst the groups of a map channel, or that it remains stable across versions.

    Always use owner_id (together with group_type) to identify an owner reliably.

    Always NULL for the kernel-global and shared group types.

value_type
    Effective value type of the group: signed-int-32 or signed-int-64.

part_id
    0-based CPU ID, or NULL for the shared group type (the shared counters have no per-partition decomposition).

key
    Counter key (a string as of LTTng 2.16.0).

value
    Counter value.

has_overflow
    1 if at least one update of the counter wrapped around (modular arithmetic) because the result couldn’t be represented with the effective value type of the map group (see the --value-type option of lttng-add-map-channel(1)), or 0 otherwise.

Note

    The schema is stable for a given format name.

    Any future incompatible change to the schema would introduce a new --format value so that existing consumers keep working.

OPTIONS

See lttng(1) for GENERAL OPTIONS.

  Recording target

-s SESSION, --session=SESSION
    Export the counter values of the recording session named SESSION instead of the current recording session.

  Output

-f FORMAT, --format=FORMAT
    Set the output format to FORMAT.

    As of LTTng 2.16.0, FORMAT must be sqlite (the default): an SQL script for the SQLite database engine (see the “SQL SCHEMA” section above).

    Important
        The sqlite format targets SQLite specifically: to quote map channel, owner, and key names, the script relies on SQLite’s string literal syntax, where the backslash is an ordinary character and the only way to write a single quote is to double it. LTTng therefore only guarantees that the script loads correctly with SQLite 3.

        You may still load the script with another SQL database engine, provided that you make that engine treat the backslash as an ordinary character within string literals (otherwise a name containing a backslash can break the quoting), for example:

        PostgreSQL
            Loads the script as is when the standard_conforming_strings setting is on, which is the default since PostgreSQL 9.1.

        MySQL and MariaDB
            Set the NO_BACKSLASH_ESCAPES SQL mode before loading the script, for example with

                SET sql_mode = 'NO_BACKSLASH_ESCAPES';

        Even then, LTTng makes no guarantee: a future format dedicated to another engine would use its own FORMAT value.

  Program information

-h, --help
    Show help.

    This option attempts to launch /usr/bin/man to view this manual page. Override the manual pager path with the LTTNG_MAN_BIN_PATH environment variable.

--list-options
    List available command options and quit.

EXIT STATUS

0
    Success

1
    Command error

2
    Undefined command

3
    Fatal error

4
    Command warning (something went wrong during the command)

ENVIRONMENT

LTTNG_ABORT_ON_ERROR
    Set to 1 to abort the process after the first error is encountered.

LTTNG_HOME
    Path to the LTTng home directory.

    Defaults to $HOME.

    Useful when the Unix user running the commands has a non-writable home directory.

LTTNG_LIST_LEGACY
    Set to 1 to use the legacy output format (LTTng 2.14 and earlier) for the lttng-list(1) command instead of the modern output format.

    Note that the legacy output doesn’t show anything related to features introduced after LTTng 2.14.

LTTNG_MAN_BIN_PATH
    Absolute path to the manual pager to use to read the LTTng command-line help (with lttng-help(1) or with the --help option) instead of /usr/bin/man.

LTTNG_NO_UTF_8
    Set to 1 to NOT emit multi-byte UTF-8 sequences, even if the locale claims to support it.

LTTNG_SESSION_CONFIG_XSD_PATH
    Path to the directory containing the session.xsd recording session configuration XML schema.

LTTNG_SESSIOND_PATH
    Absolute path to the LTTng session daemon binary (see lttng-sessiond(8)) to spawn from the lttng-create(1) command.

    The --sessiond-path general option overrides this environment variable.

LTTNG_TERM_COLOR
    Controls when to emit terminal SGR codes in the output.

    The NO_COLOR environment variable overrides this.

    One of:

    auto (default)
        Only emit SGR codes when the standard output is connected to a color-capable terminal.

    always
        Always emit SGR codes.

    never
        Never emit SGR codes.

NO_COLOR
    If set and not empty, then it’s equivalent to setting LTTNG_TERM_COLOR to never.

    See NO_COLOR <https://no-color.org/> to learn more.

FILES

$LTTNG_HOME/.lttngrc
    Unix user’s LTTng runtime configuration.

    This is where LTTng stores the name of the Unix user’s current recording session between executions of lttng(1). lttng-create(1) and lttng-set-session(1) set the current recording session.

$LTTNG_HOME/lttng-traces
    Default output directory of LTTng traces in local and snapshot modes.

    Override this path with the --output option of the lttng-create(1) command.

$LTTNG_HOME/.lttng
    Unix user’s LTTng runtime and configuration directory.

$LTTNG_HOME/.lttng/sessions
    Default directory containing the Unix user’s saved recording session configurations (see lttng-save(1) and lttng-load(1)).

/usr/local/etc/lttng/sessions
    Directory containing the system-wide saved recording session configurations (see lttng-save(1) and lttng-load(1)).

Note

    $LTTNG_HOME defaults to the value of the HOME environment variable.

EXAMPLES

Example 1. Export the maps of the current recording session and open an interactive SQL shell.

    The export becomes the initialization script of sqlite3, which then drops to its interactive prompt with the data loaded:

        $ sqlite3 -init <(lttng export-maps) :memory:

Example 2. Export the maps of a specific recording session.

    See the --session option.

        $ lttng export-maps --session=my-session > maps.sql

Example 3. Show the total count per key across all CPUs of a specific map channel.

        $ { lttng export-maps; cat << EOF
            SELECT key, SUM(value) AS total FROM vmap
            WHERE channel_name = 'my-counters'
            GROUP BY key
            ORDER BY total DESC;
          EOF
          } | sqlite3 :memory:

Example 4. Show the top 10 “hottest” counters of a specific map channel.

        $ { lttng export-maps; cat << EOF
            SELECT key, SUM(value) AS total FROM vmap
            WHERE channel_name = 'my-counters'
            GROUP BY key
            ORDER BY total DESC
            LIMIT 10;
          EOF
          } | sqlite3 :memory:

Example 5. Show the per-CPU breakdown of a specific counter of a specific map channel.

        $ { lttng export-maps; cat << EOF
            SELECT part_id, value FROM vmap
            WHERE channel_name = 'my-counters' AND key = 'myapp:my_event'
            ORDER BY part_id;
          EOF
          } | sqlite3 :memory:

Example 6. Show the total count per scheduling event of a specific Linux kernel map channel.

        $ { lttng export-maps; cat << EOF
            SELECT key, SUM(value) AS total FROM vmap
            WHERE channel_name = 'my-counters'
              AND group_type = 'kernel-global'
              AND key GLOB 'sched_*'
            GROUP BY key;
          EOF
          } | sqlite3 :memory:

Example 7. Show the top 10 processes by total count, with their command names.

    This groups by owner_id (the reliable process identifier) and shows the informational owner_name for display only.

        $ { lttng export-maps; cat << EOF
            SELECT owner_id AS pid, owner_name AS command, SUM(value) AS total
            FROM vmap
            WHERE channel_name = 'my-counters' AND group_type = 'user-per-process'
            GROUP BY owner_id
            ORDER BY total DESC
            LIMIT 10;
          EOF
          } | sqlite3 :memory:

Example 8. Show all the overflowed counters.

        $ { lttng export-maps;
            echo 'SELECT * FROM vmap WHERE has_overflow = 1;'
          } | sqlite3 :memory:

RESOURCES

    • LTTng project website <https://lttng.org>

    • LTTng documentation <https://lttng.org/docs>

    • LTTng bug tracker <https://bugs.lttng.org>

    • Git repositories <https://git.lttng.org>

    • GitHub organization <https://github.com/lttng>

    • Continuous integration <https://ci.lttng.org/>

    • Mailing list <https://lists.lttng.org/> for support and development: lttng-dev@lists.lttng.org

    • IRC channel <irc://irc.oftc.net/lttng>: #lttng on irc.oftc.net

    • Mastodon <https://mastodon.social/@lttng>

COPYRIGHT

This program is part of the LTTng-tools project.

LTTng-tools is distributed under the GNU General Public License version 2 <http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html>. See the LICENSE <https://github.com/lttng/lttng-tools/blob/master/LICENSE> file for details.

THANKS

Special thanks to Michel Dagenais and the DORSAL laboratory <http://www.dorsal.polymtl.ca/> at École Polytechnique de Montréal for the LTTng journey.

Also thanks to the Ericsson teams working on tracing which helped us greatly with detailed bug reports and unusual test cases.

SEE ALSO

lttng(1), lttng-add-map-channel(1), lttng-add-trigger(1), lttng-show-maps(1), lttng-concepts(7)