Introspection Functions

    Warning

    These functions are slow and may impose security considerations.

    For proper operation of introspection functions:

    ClickHouse saves profiler reports to the system table. Make sure the table and profiler are configured properly.

    Converts virtual memory address inside ClickHouse server process to the filename and the line number in ClickHouse source code.

    If you use official ClickHouse packages, you need to install the clickhouse-common-static-dbg package.

    Syntax

    1. addressToLine(address_of_binary_instruction)

    Arguments

    • address_of_binary_instruction (UInt64) — Address of instruction in a running process.

    Returned value

    • Source code filename and the line number in this file delimited by colon.

      1. For example, `/build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:199`, where `199` is a line number.

    • Name of a binary, if the function couldn’t find the debug information.

    • Empty string, if the address is not valid.

    Type: .

    Example

    Enabling introspection functions:

    1. SET allow_introspection_functions=1

    Selecting the first string from the trace_log system table:

    1. SELECT * FROM system.trace_log LIMIT 1 \G
    1. Row 1:
    2. ──────
    3. event_date:              2019-11-19
    4. event_time:              2019-11-19 18:57:23
    5. revision:                54429
    6. timer_type:              Real
    7. thread_number:           48
    8. query_id:                421b6855-1858-45a5-8f37-f383409d6d72
    9. trace:                   [140658411141617,94784174532828,94784076370703,94784076372094,94784076361020,94784175007680,140658411116251,140658403895439]

    The trace field contains the stack trace at the moment of sampling.

    1. SELECT addressToLine(94784076370703) \G
    1. Row 1:
    2. ──────
    3. addressToLine(94784076370703): /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:199

    Applying the function to the whole stack trace:

    1. SELECT
    2.     arrayStringConcat(arrayMap(x -> addressToLine(x), trace), '\n') AS trace_source_code_lines
    3. FROM system.trace_log
    4. LIMIT 1
    5. \G

    The arrayMap function allows to process each individual element of the trace array by the addressToLine function. The result of this processing you see in the trace_source_code_lines column of output.

    1. Row 1:
    2. ──────
    3. trace_source_code_lines: /lib/x86_64-linux-gnu/libpthread-2.27.so
    4. /usr/lib/debug/usr/bin/clickhouse
    5. /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:199
    6. /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.h:155
    7. /usr/include/c++/9/bits/atomic_base.h:551
    8. /usr/lib/debug/usr/bin/clickhouse
    9. /lib/x86_64-linux-gnu/libpthread-2.27.so
    10. /build/glibc-OTsEL5/glibc-2.27/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:97

    Converts virtual memory address inside ClickHouse server process to the symbol from ClickHouse object files.

    Syntax

    Arguments

    • address_of_binary_instruction () — Address of instruction in a running process.

    Returned value

    • Symbol from ClickHouse object files.
    • Empty string, if the address is not valid.

    Type: String.

    Example

    Enabling introspection functions:

    1. SET allow_introspection_functions=1

    Selecting the first string from the trace_log system table:

      1. Row 1:
      2. ──────
      3. event_date:    2019-11-20
      4. event_time:    2019-11-20 16:57:59
      6. timer_type:    Real
      7. thread_number: 48
      8. query_id:      724028bf-f550-45aa-910d-2af6212b94ac
      9. trace:         [94138803686098,94138815010911,94138815096522,94138815101224,94138815102091,94138814222988,94138806823642,94138814457211,94138806823642,94138814457211,94138806823642,94138806795179,94138806796144,94138753770094,94138753771646,94138753760572,94138852407232,140399185266395,140399178045583]

      The trace field contains the stack trace at the moment of sampling.

      Getting a symbol for a single address:

      1. SELECT addressToSymbol(94138803686098) \G
      1. Row 1:
      2. ──────
      3. addressToSymbol(94138803686098): _ZNK2DB24IAggregateFunctionHelperINS_20AggregateFunctionSumImmNS_24AggregateFunctionSumDataImEEEEE19addBatchSinglePlaceEmPcPPKNS_7IColumnEPNS_5ArenaE

      Applying the function to the whole stack trace:

      1. SELECT
      2.     arrayStringConcat(arrayMap(x -> addressToSymbol(x), trace), '\n') AS trace_symbols
      3. FROM system.trace_log
      4. LIMIT 1
      5. \G

      The function allows to process each individual element of the trace array by the addressToSymbols function. The result of this processing you see in the trace_symbols column of output.

      1. Row 1:
      2. ──────
      3. trace_symbols: _ZNK2DB24IAggregateFunctionHelperINS_20AggregateFunctionSumImmNS_24AggregateFunctionSumDataImEEEEE19addBatchSinglePlaceEmPcPPKNS_7IColumnEPNS_5ArenaE
      4. _ZNK2DB10Aggregator21executeWithoutKeyImplERPcmPNS0_28AggregateFunctionInstructionEPNS_5ArenaE
      5. _ZN2DB10Aggregator14executeOnBlockESt6vectorIN3COWINS_7IColumnEE13immutable_ptrIS3_EESaIS6_EEmRNS_22AggregatedDataVariantsERS1_IPKS3_SaISC_EERS1_ISE_SaISE_EERb
      6. _ZN2DB10Aggregator14executeOnBlockERKNS_5BlockERNS_22AggregatedDataVariantsERSt6vectorIPKNS_7IColumnESaIS9_EERS6_ISB_SaISB_EERb
      7. _ZN2DB10Aggregator7executeERKSt10shared_ptrINS_17IBlockInputStreamEERNS_22AggregatedDataVariantsE
      8. _ZN2DB27AggregatingBlockInputStream8readImplEv
      9. _ZN2DB17IBlockInputStream4readEv
      10. _ZN2DB26ExpressionBlockInputStream8readImplEv
      11. _ZN2DB17IBlockInputStream4readEv
      12. _ZN2DB26ExpressionBlockInputStream8readImplEv
      13. _ZN2DB17IBlockInputStream4readEv
      14. _ZN2DB28AsynchronousBlockInputStream9calculateEv
      15. _ZNSt17_Function_handlerIFvvEZN2DB28AsynchronousBlockInputStream4nextEvEUlvE_E9_M_invokeERKSt9_Any_data
      16. _ZN14ThreadPoolImplI20ThreadFromGlobalPoolE6workerESt14_List_iteratorIS0_E
      17. _ZZN20ThreadFromGlobalPoolC4IZN14ThreadPoolImplIS_E12scheduleImplIvEET_St8functionIFvvEEiSt8optionalImEEUlvE1_JEEEOS4_DpOT0_ENKUlvE_clEv
      18. _ZN14ThreadPoolImplISt6threadE6workerESt14_List_iteratorIS0_E
      19. execute_native_thread_routine
      20. start_thread
      21. clone

      Converts a symbol that you can get using the addressToSymbol function to the C++ function name.

      Syntax

      1. demangle(symbol)

      Arguments

      • symbol () — Symbol from an object file.

      Returned value

      • Name of the C++ function.
      • Empty string if a symbol is not valid.

      Type: String.

      Enabling introspection functions:

      1. SET allow_introspection_functions=1

      Selecting the first string from the trace_log system table:

      1. Row 1:
      2. ──────
      3. event_date:    2019-11-20
      4. revision:      54429
      6. thread_number: 48
      7. query_id:      724028bf-f550-45aa-910d-2af6212b94ac
      8. trace:         [94138803686098,94138815010911,94138815096522,94138815101224,94138815102091,94138814222988,94138806823642,94138814457211,94138806823642,94138814457211,94138806823642,94138806795179,94138806796144,94138753770094,94138753771646,94138753760572,94138852407232,140399185266395,140399178045583]

      The trace field contains the stack trace at the moment of sampling.

      Getting a function name for a single address:

      1. SELECT demangle(addressToSymbol(94138803686098)) \G
      1. Row 1:
      2. ──────
      3. demangle(addressToSymbol(94138803686098)): DB::IAggregateFunctionHelper<DB::AggregateFunctionSum<unsigned long, unsigned long, DB::AggregateFunctionSumData<unsigned long> > >::addBatchSinglePlace(unsigned long, char*, DB::IColumn const**, DB::Arena*) const

      Applying the function to the whole stack trace:

      1. SELECT
      2.     arrayStringConcat(arrayMap(x -> demangle(addressToSymbol(x)), trace), '\n') AS trace_functions
      3. FROM system.trace_log
      4. LIMIT 1
      5. \G

      The function allows to process each individual element of the trace array by the demangle function. The result of this processing you see in the trace_functions column of output.

      1. Row 1:
      2. ──────
      3. trace_functions: DB::IAggregateFunctionHelper<DB::AggregateFunctionSum<unsigned long, unsigned long, DB::AggregateFunctionSumData<unsigned long> > >::addBatchSinglePlace(unsigned long, char*, DB::IColumn const**, DB::Arena*) const
      4. DB::Aggregator::executeWithoutKeyImpl(char*&, unsigned long, DB::Aggregator::AggregateFunctionInstruction*, DB::Arena*) const
      5. DB::Aggregator::executeOnBlock(std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > >, unsigned long, DB::AggregatedDataVariants&, std::vector<DB::IColumn const*, std::allocator<DB::IColumn const*> >&, std::vector<std::vector<DB::IColumn const*, std::allocator<DB::IColumn const*> >, std::allocator<std::vector<DB::IColumn const*, std::allocator<DB::IColumn const*> > > >&, bool&)
      6. DB::Aggregator::executeOnBlock(DB::Block const&, DB::AggregatedDataVariants&, std::vector<DB::IColumn const*, std::allocator<DB::IColumn const*> >&, std::vector<std::vector<DB::IColumn const*, std::allocator<DB::IColumn const*> >, std::allocator<std::vector<DB::IColumn const*, std::allocator<DB::IColumn const*> > > >&, bool&)
      7. DB::Aggregator::execute(std::shared_ptr<DB::IBlockInputStream> const&, DB::AggregatedDataVariants&)
      8. DB::AggregatingBlockInputStream::readImpl()
      9. DB::IBlockInputStream::read()
      10. DB::ExpressionBlockInputStream::readImpl()
      11. DB::IBlockInputStream::read()
      12. DB::ExpressionBlockInputStream::readImpl()
      13. DB::IBlockInputStream::read()
      14. DB::AsynchronousBlockInputStream::calculate()
      15. std::_Function_handler<void (), DB::AsynchronousBlockInputStream::next()::{lambda()#1}>::_M_invoke(std::_Any_data const&)
      16. ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::_List_iterator<ThreadFromGlobalPool>)
      17. ThreadFromGlobalPool::ThreadFromGlobalPool<ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::function<void ()>, int, std::optional<unsigned long>)::{lambda()#3}>(ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::function<void ()>, int, std::optional<unsigned long>)::{lambda()#3}&&)::{lambda()#1}::operator()() const
      18. ThreadPoolImpl<std::thread>::worker(std::_List_iterator<std::thread>)
      19. execute_native_thread_routine
      20. start_thread
      21. clone

      Returns id of the thread, in which current Block is processed.

      Syntax

      1. tid()

      Returned value

      • Current thread id. .

      Example

      Query:

      1. SELECT tid();

      Result:

      1. ┌─tid()─┐
      2. │  3878 │
      3. └───────┘

      Emits trace log message to server log for each Block.

      Syntax

      1. logTrace('message')

      Arguments

      • message — Message that is emitted to server log. .

      Returned value

      • Always returns 0.

      Example

      Query:

      Result:

      1. ┌─logTrace('logTrace message')─┐
      2. │                            0 │
      3. └──────────────────────────────┘