Using perf

    To capture some data with call graphs:

    1. sudo perf record -p `pidof ceph-osd` -F 99 --call-graph dwarf -- sleep 60

    To view by caller (where you can see what each top function calls):

      • note
      • If the caller/callee views look the same you may besuffering from a kernel bug; upgrade to 4.8 or later.

      Ceph use RelWithDebInfo as its default CMAKE_BUILD_TYPE. Hence -O2 -g isused to compile the tree in this case. And the -O2 optimization levelenables -fomit-frame-pointer by default. But this prevents stack profilersfrom accessing the complete stack information. So one can disable this optionwhen launching cmake

      1. cmake -DCMAKE_CXX_FLAGS="-fno-omit-frame-pointer"

      or when building the tree:

        Flamegraphs

        Run ceph, then record some perf data:

        1. sudo perf record -p `pidof ceph-osd` -F 99 --call-graph dwarf -- sleep 60

        Then generate the flamegraph:

        1. sudo perf script | ~/src/FlameGraph/stackcollapse-perf.pl > /tmp/folded
        2. ~/src/FlameGraph/flamegraph.pl /tmp/folded > /tmp/perf.svg