官方10年前开始就一直在致力于优化器代码的重构工作,目的是能确保在SQL的执行过程中有清晰的阶段,包括分离Parse和Resolve阶段、更好的支持更多新的语法、保证Name和Type的解析、尽量在Prepare阶段做完Transformations,这样能够很容易的支持CTEs、Windows函数、LATERAL语法、JSON的Table函数和Windows函数。当然,最重要的重构当数Iterator执行器。这个早期MySQL版本的优化器执行器逻辑:

2.png

不过本文还要普及一些MySQL基础的知识。

首先我们先了解Parse后的AST树到底是长什么样子?先了解重要的两个结构: SELECT_LEX & SELECT_LEX_UNIT

SELECT_LEX: 代表了SELECT本身,也就是SQL SELECT的关键字就会对应一个SELECT_LEX的结构。 SELECT_LEX_UNIT: 代表了带UNION的一堆SELECT,当然也可以代表一个SELECT。

下面这段是SELECT_LEX_UNIT/SELECT_LEX的类结构中,和查询层次相关的一些成员变量

我们来拿一个真实的例子来举例说明:

  1. FROM ttt1)
  2. UNION ALL
  3. (SELECT *
  4. FROM
  5. (SELECT *
  6. FROM ttt2) AS a,
  7. (SELECT *
  8. FROM ttt3
  9. UNION ALL SELECT *
  10. FROM ttt4) AS b)

实际中该查询的标准内存结构就是这样的,

这里需要注意的是,MySQL官方的 [WL#5275: Process subqueries in FROM clause in the same way as] view支持可以把子查询提升到上层查询中。优化器会调用SELECT_LEX::resolve_placeholder_tables -> SELECT_LEX::merge_derived来避免materialize那些可以提升到上层查询的子查询。外部也可以通过set optimizer_switch=’derived_merge=on/off’来进行开关,下面来对比下8.0.13和8.0.18对于该优化的执行计划展现。

  1. 8.0.13 derived_merge off vs on
  2. +----+-------------+------------+------------+------+---------------+------+---------+------+------+----------+---------------------------------------+
  3. | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
  4. +----+-------------+------------+------------+------+---------------+------+---------+------+------+----------+---------------------------------------+
  5. | 1 | PRIMARY | ttt1 | NULL | ALL | NULL | NULL | NULL | NULL | 1 | 100.00 | NULL |
  6. | 2 | UNION | <derived3> | NULL | ALL | NULL | NULL | NULL | NULL | 2 | 100.00 | NULL |
  7. | 2 | UNION | <derived4> | NULL | ALL | NULL | NULL | NULL | NULL | 4 | 100.00 | Using join buffer (Block Nested Loop) |
  8. | 4 | DERIVED | ttt3 | NULL | ALL | NULL | NULL | NULL | NULL | 1 | 100.00 | NULL |
  9. | 5 | UNION | ttt4 | NULL | ALL | NULL | NULL | NULL | NULL | 1 | 100.00 | NULL |
  10. | 3 | DERIVED | ttt2 | NULL | ALL | NULL | NULL | NULL | NULL | 1 | 100.00 | NULL |
  11. +----+-------------+------------+------------+------+---------------+------+---------+------+------+----------+---------------------------------------+
  12. 6 rows in set, 1 warning (0.01 sec)
  13. +----+-------------+------------+------------+------+---------------+------+---------+------+------+----------+---------------------------------------+
  14. | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
  15. +----+-------------+------------+------------+------+---------------+------+---------+------+------+----------+---------------------------------------+
  16. | 1 | PRIMARY | ttt1 | NULL | ALL | NULL | NULL | NULL | NULL | 1 | 100.00 | NULL |
  17. | 2 | UNION | ttt2 | NULL | ALL | NULL | NULL | NULL | NULL | 1 | 100.00 | NULL |
  18. | 2 | UNION | <derived4> | NULL | ALL | NULL | NULL | NULL | NULL | 4 | 100.00 | Using join buffer (Block Nested Loop) |
  19. | 4 | DERIVED | ttt3 | NULL | ALL | NULL | NULL | NULL | NULL | 1 | 100.00 | NULL |
  20. | 5 | UNION | ttt4 | NULL | ALL | NULL | NULL | NULL | NULL | 1 | 100.00 | NULL |
  21. +----+-------------+------------+------------+------+---------------+------+---------+------+------+----------+---------------------------------------+
  22. 5 rows in set, 1 warning (0.02 sec)
  23. 8.0.18 derived_merge off vs on
  24. | -> Append
  25. -> Stream results
  26. -> Table scan on ttt1 (cost=0.35 rows=1)
  27. -> Stream results
  28. -> Inner hash join
  29. -> Table scan on b
  30. -> Union materialize
  31. -> Table scan on ttt3 (cost=0.35 rows=1)
  32. -> Table scan on ttt4 (cost=0.35 rows=1)
  33. -> Hash
  34. -> Table scan on a
  35. -> Materialize
  36. -> Table scan on ttt2 (cost=0.35 rows=1)
  37. | -> Append
  38. -> Stream results
  39. -> Table scan on ttt1 (cost=0.35 rows=1)
  40. -> Stream results
  41. -> Inner hash join
  42. -> Table scan on b
  43. -> Union materialize
  44. -> Table scan on ttt3 (cost=0.35 rows=1)
  45. -> Table scan on ttt4 (cost=0.35 rows=1)
  46. -> Hash
  47. -> Table scan on ttt2 (cost=0.35 rows=1)

通过优化后,该查询的内存结构就变成这样了 4.png

本文由于不是介绍整个优化器的详细优化过程,所以我们这里简单介绍下优化器的一些步骤和方法,根据MySQL官方网站的介绍我们可以知道具体步骤如下:

  1. handle_select()
  2. mysql_select()
  3. JOIN::prepare()
  4. setup_fields()
  5. JOIN::optimize() /* optimizer is from here ... */
  6. optimize_cond()
  7. opt_sum_query()
  8. make_join_statistics()
  9. get_quick_record_count()
  10. choose_plan()
  11. /* Find the best way to access tables */
  12. /* as specified by the user. */
  13. optimize_straight_join()
  14. best_access_path()
  15. /* Find a (sub-)optimal plan among all or subset */
  16. /* of all possible query plans where the user */
  17. /* controls the exhaustiveness of the search. */
  18. greedy_search()
  19. best_extension_by_limited_search()
  20. best_access_path()
  21. /* Perform an exhaustive search for an optimal plan */
  22. find_best()
  23. make_join_select() /* ... to here */
  24. JOIN::exec()

  1. Class Sql_cmd_dml
  2. Sql_cmd_dml::prepare() walks through these common steps:
  3. precheck() - performs a coarse-grained authorization check of the statement.
  4. open_tables_for_query() - opens the tables and views mentioned in the statement. Views are expanded so that underlying views and tables are opened too.
  5. resolve_var_assignments() - resolves variable assignments in the statement.
  6. prepare_inner() - performs statement-specific preparation of the statement and is implemented for every subclass of Sql_cmd_dml.
  7. Sql_cmd_dml::execute() walks through these common steps:
  8. set_statement_timer() - is called if a time limit is applicable to the statement.
  9. prepare() is called if the statement is a regular (not preparable) statement.
  10. If prepare() is not called, precheck() and open_tables_for_query() are still called since these actions are required also when executing already prepared statements.
  11. run_before_dml_hook() is called if the statement is a data change statement, in order to prepare replication actions for the statement.
  12. An IGNORE or STRICT mode error handler is set up if applicable. It will be active for the duration of the execution of the statement.
  13. lock_tables() is called, unless the statement affects no rows or produces no rows in any tables.
  14. Query_cache::store_query() is called to register the statement in the query cache, if applicable.
  15. execute_inner() - performs statement-specific optimization and execution of the statement. Sql_cmd_dml::execute_inner() is an implementation for all SELECT statements, all INSERT statements that are based on a SELECT and all multi-table UPDATE and DELETE statements (ie all statements that are implemented using a JOIN object). For all other types of DML statements, a separate implementation for execute_inner() exists.
  16. Class Sql_cmd_select
  17. This is a new class used to implement SELECT statements.
  18. It has an implementation of prepare_inner() to prepare SELECT statements.
  19. It uses Sql_cmd_dml::execute_inner() to execute SELECT statements.

Sql_cmd_dml是LEX的成员变量m_sql_cmd,而lex->m_sql_cmd大部分会在sql/sql_yacc.yy中new出来,所以目前8.0.13版本整个的流程就变成了下面的流程

  1. 8.0.13
  2. mysql_execute_command()
  3. lex->m_sql_cmd->execute()
  4. Sql_cmd_dml::execute()
  5. Sql_cmd_dml::prepare()
  6. Sql_cmd_select::precheck()
  7. Sql_cmd_select::open_tables_for_query()
  8. Sql_cmd_select::prepare_inner()
  9. SELECT_LEX_UNIT::prepare_limit()
  10. SELECT_LEX_UNIT::prepare() (not simple or simple SELECT_LEX::prepare)
  11. SELECT_LEX::prepare()
  12. ......
  13. Sql_cmd_dml::execute_inner
  14. SELECT_LEX_UNIT::optimize() (not simple or simple SELECT_LEX::optimize)
  15. SELECT_LEX::optimize()
  16. JOIN::optimize()
  17. SELECT_LEX_UNIT::optimize()
  18. ......
  19. SELECT_LEX_UNIT::execute() (not simple or simple SELECT_LEX::optimize)
  20. SELECT_LEX::execute()
  21. JOIN::exec()
  22. JOIN::prepare_result()
  23. do_select()
  24. sub_select()
  25. ......
  26. SELECT_LEX_UNIT::execute()
  27. ......
  28. SELECT_LEX_UNIT::cleanup(false)

打开set debug=”+d,info,error,query,enter,general,where:O,/tmp/mysqld.trace”可以看到更详细的执行步骤

  1. T@8: | | | | | | | >do_select
  2. T@8: | | | | | | | | >sub_select
  3. T@8: | | | | | | | | | >innobase_trx_init
  4. T@8: | | | | | | | | | <innobase_trx_init 3269
  5. T@8: | | | | | | | | | >handler::ha_index_init
  6. T@8: | | | | | | | | | | >index_init
  7. T@8: | | | | | | | | | | <index_init 10243
  8. T@8: | | | | | | | | | | >change_active_index
  9. T@8: | | | | | | | | | | | >innobase_get_index
  10. T@8: | | | | | | | | | | | <innobase_get_index 11071
  11. T@8: | | | | | | | | | | <change_active_index 11172
  12. T@8: | | | | | | | | | <handler::ha_index_init 2793
  13. T@8: | | | | | | | | | >handler::ha_index_first
  14. T@8: | | | | | | | | | | >index_first
  15. T@8: | | | | | | | | | | | >index_read
  16. T@8: | | | | | | | | | | | | >row_search_mvcc
  17. T@8: | | | | | | | | | | | | | >row_sel_store_mysql_rec
  18. T@8: | | | | | | | | | | | | | | >row_sel_store_mysql_field_func
  19. T@8: | | | | | | | | | | | | | | <row_sel_store_mysql_field_func 2921
  20. T@8: | | | | | | | | | | | | | <row_sel_store_mysql_rec 3080
  21. T@8: | | | | | | | | | | | | <row_search_mvcc 5881
  22. T@8: | | | | | | | | | | | <index_read 11012
  23. T@8: | | | | | | | | | <handler::ha_index_first 3293
  24. T@8: | | | | | | | | | >evaluate_join_record
  25. T@8: | | | | | | | | | | enter: join: 0x7fff99d92d68 join_tab index: 0 table: cat cond: (nil)
  26. T@8: | | | | | | | | | | >sub_select_op

这里不在赘述这个stack是因为,我们下面要引入了我们重要的主题部分8.0.18的Iterator执行部分,看看这个与之前的执行有何不同。官方用了很多Worklogs来实现Iterator的执行器。

先来了解下术语: QEP:全称(Query Execution Plan)查询执行计划。 QEP_TAB:全称(Query Execution Plan Table) 查询执行计划表 熟悉我们知道在8.0开始,官方已经慢慢的用Iterator的执行类来替换原有的一些和执行相关的类,所以原有的流程中bool JOIN::optimize(),用于生成一个Query块的执行计划(QEP)就增加了生成Iterator的部分。

6.png

最终要的JOIN::create_iterators主要分两个步骤: 1) 通过create_table_iterators,对于每个特定表,生成对应的基本的RowIterators的继承子类。 2) 通过调用create_root_iterator_for_join,生成组合的iterators,合并每个表的组合行。

然后将生成的iterator赋值到JOIN::m_root_iterator。

JOIN::create_table_iterators里面可以看到需要去轮询所有的表来构建访问方式,调用了最重要的方法QEP_TAB::make_join_readinfo和QEP_TAB::pick_table_access_method,我们来看看和之前非Iterator访问方式有何不同。 在8.0之前,我们看到QEP_TAB是通过一些函数指针和READ_RECORD来设定访问的函数指针:

执行的流程如下:

  1. if (in_first_read)
  2. {
  3. in_first_read= false;
  4. error= (*qep_tab->read_first_record)(qep_tab); //设定合适的读取函数,如设定索引读函数/全表扫描函数
  5. }
  6. else
  7. error= info->read_record(info);

那么对于第一次QEP_TAB::read_first_record和后续读指针READ_RECORD::read_record可以为下列函数的实现,其中rr代表read record:

  1. int join_init_quick_read_record(QEP_TAB *tab);
  2. int join_init_read_record(QEP_TAB *tab);
  3. int join_read_first(QEP_TAB *tab);
  4. int join_read_last(QEP_TAB *tab);
  5. int join_read_last_key(QEP_TAB *tab);join_read_next_same
  6. int join_materialize_derived(QEP_TAB *tab);
  7. int join_materialize_semijoin(QEP_TAB *tab);
  8. int join_read_prev_same(READ_RECORD *info);
  9. static int join_read_const(QEP_TAB *tab);
  10. static int read_const(TABLE *table, TABLE_REF *ref);
  11. static int join_read_key(QEP_TAB *tab);
  12. static int join_read_always_key(QEP_TAB *tab);
  13. static int join_no_more_records(READ_RECORD *info);
  14. static int join_read_next(READ_RECORD *info);
  15. static int join_read_next_same(READ_RECORD *info);
  16. static int join_read_prev(READ_RECORD *info);
  17. static int join_ft_read_first(QEP_TAB *tab);
  18. static int join_ft_read_next(READ_RECORD *info);
  19. static int join_read_always_key_or_null(QEP_TAB *tab);
  20. static int join_read_next_same_or_null(READ_RECORD *info);
  21. int rr_sequential(READ_RECORD *info)
  22. static int rr_quick(READ_RECORD *info);
  23. int rr_sequential(READ_RECORD *info);
  24. static int rr_from_tempfile(READ_RECORD *info);
  25. template<bool> static int rr_unpack_from_tempfile(READ_RECORD *info);
  26. template<bool> static int rr_unpack_from_buffer(READ_RECORD *info);
  27. static int rr_from_pointers(READ_RECORD *info);
  28. static int rr_from_cache(READ_RECORD *info);
  29. static int init_rr_cache(THD *thd, READ_RECORD *info);
  30. static int rr_index_first(READ_RECORD *info);
  31. static int rr_index_last(READ_RECORD *info);
  32. static int rr_index(READ_RECORD *info);
  33. static int rr_index_desc(READ_RECORD *info);

为什么简单的流程,需要指定不同的函数指针呢?原因是因为优化器需要根据不同的规则(RBO)和代价(CBO)去设计巧妙的访问方法,比如表扫描、索引扫描、稀疏扫描等等,那么这样的组合对于Innodb层提供的简单接口来说非常复杂。Innodb层和Server层的接口也不会根据上层的变化不断的修改和增加,所以Server层的执行层,利用自己规定的方法,来进行组合调用。比如我们举例rr_quick函数。

  1. static int rr_quick(READ_RECORD *info)
  2. {
  3. int tmp;
  4. while ((tmp= info->quick->get_next()))
  5. {
  6. if (info->thd->killed || (tmp != HA_ERR_RECORD_DELETED))
  7. {
  8. tmp= rr_handle_error(info, tmp);
  9. break;
  10. }
  11. }
  12. return tmp;
  13. }

rr_quick增加了一个新的优化的类就是QUICK_SELECT_I接口实现的具体优化类,顾名思义就是比表扫描和索引扫描快速的访问方式,目前官方有7种方式,

  1. enum {
  2. QS_TYPE_RANGE = 0,
  3. QS_TYPE_INDEX_MERGE = 1,
  4. QS_TYPE_RANGE_DESC = 2,
  5. QS_TYPE_FULLTEXT = 3,
  6. QS_TYPE_ROR_INTERSECT = 4,
  7. QS_TYPE_ROR_UNION = 5,
  8. QS_TYPE_GROUP_MIN_MAX = 6
  9. };
  1. 1. Create quick select
  2. quick= new QUICK_XXX_SELECT(...);
  3. 2. Perform lightweight initialization. This can be done in 2 ways:
  4. 2.a: Regular initialization
  5. if (quick->init())
  6. {
  7. //the only valid action after failed init() call is delete
  8. delete quick;
  9. }
  10. 2.b: Special initialization for quick selects merged by QUICK_ROR_*_SELECT
  11. if (quick->init_ror_merged_scan())
  12. delete quick;
  13. 3. Perform zero, one, or more scans.
  14. while (...)
  15. {
  16. // initialize quick select for scan. This may allocate
  17. // buffers and/or prefetch rows.
  18. if (quick->reset())
  19. {
  20. //the only valid action after failed reset() call is delete
  21. delete quick;
  22. //abort query
  23. }
  24. // perform the scan
  25. do
  26. {
  27. res= quick->get_next();
  28. } while (res && ...)
  29. }
  30. 4. Delete the select:
  31. delete quick;

显然,rr_quick仍然是执行路径分类下的又一个复杂的路由函数,根据实际READ_RECORD::quick的具体QUICK class来决定剩余的逻辑,那如何对应到Innodb存储的具体函数呢?拿QUICK_RANGE_SELECT这个类来举例,参照如下调用stack:

  1. #x ha_index_first/ha_index_read_map or ha_index_next_same/ha_index_next
  2. #0 handler::read_range_first or handler::read_range_next
  3. #1 handler::multi_range_read_next (this=0x7f9a78080900, range_info=0x7f9adc38bd40)
  4. #2 DsMrr_impl::dsmrr_next (this=0x7f9a78082628, range_info=0x7f9adc38bd40)
  5. #3 ha_innobase::multi_range_read_next (this=0x7f9a78080900, range_info=0x7f9adc38bd40)
  6. #4 QUICK_RANGE_SELECT::get_next (this=0x7f9a7807b220)
  7. #5 rr_quick (info=0x7f9a78103dd8)
  8. #6 join_init_read_record (tab=0x7f9a78103d48)
  9. #7 sub_select (join=0x7f9a78005bd8, join_tab=0x7f9a78103d48, end_of_records=false)
  10. #8 do_select (join=0x7f9a78005bd8)
  11. #9 JOIN::exec (this=0x7f9a78005bd8)

现在回到了我们的8.0 Iterator执行器中,我们看到READ_RECORD m_read_record_info将被unique_ptr_destroy_only m_iterator所代替,包括setup_read_record(), init_read_record() and setup_read_record_idx()都将被各种各样的Iterator代替。在Iterator的执行器下,不用关心函数指针的赋值,也不需要有两个QEP_TAB::read_first_record和后续读指针READ_RECORD::read_record,只需要实现RowIterator的子类并实现其定义的接口。

详细可以查看官方的link: https://dev.mysql.com/doc/dev/mysql-server/latest/classTableRowIterator.html

QEP_TAB::pick_table_access_method设置流程变为了下面的方式:

  1. void QEP_TAB::pick_table_access_method() {
  2. ......
  3. switch (type()) {
  4. case JT_REF:
  5. if (is_pushed_child) {
  6. DBUG_ASSERT(!m_reversed_access);
  7. iterator = NewIterator<PushedJoinRefIterator>(
  8. join()->thd, table(), &ref(), use_order(), &join()->examined_rows);
  9. } else if (m_reversed_access) {
  10. iterator = NewIterator<RefIterator<true>>(join()->thd, table(), &ref(),
  11. use_order(), this,
  12. &join()->examined_rows);
  13. } else {
  14. iterator = NewIterator<RefIterator<false>>(join()->thd, table(), &ref(),
  15. use_order(), this,
  16. &join()->examined_rows);
  17. }
  18. used_ref = &ref();
  19. break;
  20. case JT_REF_OR_NULL:
  21. iterator = NewIterator<RefOrNullIterator>(join()->thd, table(), &ref(),
  22. use_order(), this,
  23. &join()->examined_rows);
  24. used_ref = &ref();
  25. break;
  26. case JT_CONST:
  27. iterator = NewIterator<ConstIterator>(join()->thd, table(), &ref(),
  28. &join()->examined_rows);
  29. break;
  30. case JT_EQ_REF:
  31. if (is_pushed_child) {
  32. iterator = NewIterator<PushedJoinRefIterator>(
  33. join()->thd, table(), &ref(), use_order(), &join()->examined_rows);
  34. } else {
  35. iterator = NewIterator<EQRefIterator>(
  36. join()->thd, table(), &ref(), use_order(), &join()->examined_rows);
  37. }
  38. used_ref = &ref();
  39. break;
  40. ......
  41. case JT_ALL:
  42. case JT_RANGE:
  43. case JT_INDEX_MERGE:
  44. if (using_dynamic_range) {
  45. iterator = NewIterator<DynamicRangeIterator>(join()->thd, table(), this,
  46. &join()->examined_rows);
  47. } else {
  48. iterator =
  49. create_table_iterator(join()->thd, nullptr, this, false,
  50. &join()->examined_rows, &m_using_table_scan);
  51. }
  52. ......
  53. }

执行的流程变成了:

  1. unique_ptr<RowIterator> iterator(new ...);
  2. if (iterator->Init())
  3. return true;
  4. while (iterator->Read() == 0) {
  5. ...
  6. }

Join访问对比

MySQL 的 join 操作主要是采用NestLoop的算法,其中涉及的主要函数有如下 do_select()、sub_select()、evaluate_join_record(),当然还有BNL和BKA等等,我们就不再这里赘述。

  1. static int do_select(JOIN *join)
  2. {
  3. ... ...
  4. if (join->plan_is_const() && !join->need_tmp) {
  5. ... ...
  6. } else {
  7. QEP_TAB *qep_tab= join->qep_tab + join->const_tables;
  8. DBUG_ASSERT(join->primary_tables);
  9. error= join->first_select(join,qep_tab,0); 非结束选取
  10. if (error >= NESTED_LOOP_OK)
  11. error= join->first_select(join,qep_tab,1); 结束选取
  12. }
  13. }
  14. enum_nested_loop_state sub_select(JOIN *join,JOIN_TAB *join_tab,bool end_of_records)
  15. {
  16. ... ...
  17. if (end_of_records)
  18. {
  19. enum_nested_loop_state nls=
  20. (*join_tab->next_select)(join,join_tab+1,end_of_records); 一般是sub_select/最后一个是end_send/end_send_group
  21. DBUG_RETURN(nls);
  22. }
  23. READ_RECORD *info= &join_tab->read_record;
  24. ... ...
  25. while (rc == NESTED_LOOP_OK && join->return_tab >= qep_tab_idx)
  26. {
  27. int error;
  28. if (in_first_read) 读取第一条记录
  29. {
  30. in_first_read= false;
  31. error= (*qep_tab->read_first_record)(qep_tab);
  32. }
  33. else
  34. error= info->read_record(info); 循环读取记录直到结束位置
  35. ......
  36. rc= evaluate_join_record(join, qep_tab); 评估是否符合条件,连接下一个表
  37. }
  38. ... ...
  39. }
  40. static enum_nested_loop_state
  41. evaluate_join_record(JOIN *join, JOIN_TAB *join_tab)
  42. {
  43. ... ...
  44. Item *condition= join_tab->condition(); 查询条件
  45. bool found= TRUE;
  46. ... ...
  47. if (condition)
  48. {
  49. found= MY_TEST(condition->val_int()); 评估是否符合条件
  50. ... ...
  51. if (found)
  52. {
  53. enum enum_nested_loop_state rc;
  54. /* A match from join_tab is found for the current partial join. */
  55. rc= (*join_tab->next_select)(join, join_tab+1, 0);
  56. join->thd->get_stmt_da()->inc_current_row_for_warning();
  57. if (rc != NESTED_LOOP_OK)
  58. DBUG_RETURN(rc);
  59. ... ...
  60. }

那么整个执行的流程串起来就是:

  1. JOIN::exec() 执行一个Query Block
  2. |-THD_STAGE_INFO() 设置线程的状态为executing
  3. |-set_executed() 设置为执行状态,JOIN::executed=true
  4. |-prepare_result()
  5. |-send_result_set_metadata() 先将元数据发送给客户端
  6. |
  7. |-do_select() ←### 查询的实际入口函数,做JOIN操作,会返回给客户端或写入表
  8. |
  9. |-join->first_select(join,qep_tab,0) 1. 执行nest loop操作,默认会调用sub_select()函数,
  10. | | 也即循环调用rnd_next()+evaluate_join_record()
  11. | |
  12. | |###while循环读取数据###
  13. | | 2. 调用存储引擎接口读取数据
  14. | |-qep_tab->read_first_record() 2.1. 首次调用,实际为join_init_read_record()
  15. | | |-tab->quick()->reset() 对于quick调用QUICK_RANGE_SELECT::reset()函数
  16. | | | |-file->ha_index_init() 会调用存储引擎接口
  17. | | | | |-index_init()
  18. | | | | |-change_active_index()
  19. | | | | |-innobase_get_index()
  20. | | | |-file->multi_range_read_init()
  21. | | |-init_read_record() 设置read_record指针,在此为rr_quick
  22. | |
  23. | |-info->read_record() 2.2 再次调用,如上,该函数在init_read_record()中初始化
  24. | | |-info->quick->get_next() 实际调用QUICK_RANGE_SELECT::get_next()
  25. | | |-file->multi_range_read_next() 调用handler.cc文件中函数
  26. | | |-read_range_first() 对于第一次调用
  27. | | | |-ha_index_read_map() 存储引擎调用
  28. | | | |-index_read()
  29. | | | |-row_search_mvcc()
  30. | | |
  31. | | |-read_range_next() 对于非第一次调用
  32. | | |-ha_index_next()
  33. | | |-general_fetch()
  34. | | |-row_search_mvcc()
  35. | |
  36. | |-evaluate_join_record() 2.3 处理读取的记录,判断是否满足条件,包括了第一条记录
  37. | |-qep_tab->next_select() 对于查询,实际会调用end_send()
  38. | |-Query_result_send::send_data()
  39. |
  40. |-join->first_select(join,qep_tab,1) 3. 一个table已经读取数据结束,同样默认调用sub_select()
  41. | |-join_tab->next_select() 调用该函数处理下个表或者结束处理
  42. |
  43. |-join->select_lex->query_result()->send_eof()

这次我们要对比下新的执行引擎的变化,既然表的访问方式已经从函数指针变为Iterator的Init/Read两个接口,我们来看其实对于Iterator引擎更容易理解了,JOIN::create_table_iterators本身就可以构造出简单的Iterators结构:

  1. | -> Limit: 1 row(s)
  2. -> Sort: ttt1.c1, limit input to 1 row(s) per chunk (cost=0.45 rows=2)
  3. -> Filter: (ttt1.c1 > 2)
  4. -> Table scan on ttt1

而JOIN::create_root_iterator_for_join可以构造出更为标准的Iterator火山模型结构:

  1. | -> Limit: 1 row(s)
  2. -> Sort: ttt1.c1, limit input to 1 row(s) per chunk
  3. -> Stream results
  4. -> Inner hash join (ttt1.c1 = ttt2.c1) (cost=0.90 rows=1)
  5. -> Table scan on ttt1 (cost=0.45 rows=2)
  6. -> Hash
  7. -> Filter: (ttt2.c1 > 0) (cost=0.35 rows=1)
  8. -> Table scan on ttt2 (cost=0.35 rows=1)
  9. |

create_root_iterator_for_join中最为重要的函数ConnectJoins,里面负责生成相应的Semijoin/Hashjoin/Antijoin/Nestloopjoin等等的组合的Iterator。因为Hashjoin另有篇幅介绍,这里举例来说NestLoopIterator的实现:

最后我们用Hashjoin看下新的执行流程吧:

  1. SELECT_LEX_UNIT::execute() 执行一个Query Unit
  2. |-SELECT_LEX_UNIT::ExecuteIteratorQuery
  3. |-THD_STAGE_INFO() 设置线程的状态为executing
  4. |-query_result->start_execution(thd) 设置为执行状态,Query result execution_started = true;
  5. |-query_result->send_result_set_metadata() 先将元数据发送给客户端
  6. |-set_executed(); Unit executed = true;
  7. |
  8. |-m_root_iterator->Init() 所有Iterator递归Init,此处IteratorHashJoinIterator
  9. | |-HashJoinIterator::Init()
  10. | | |-TableScanIterator::Init()
  11. | | | |-handler::ha_rnd_init()
  12. |
  13. | | |-HashJoinIterator::BuildHashTable()
  14. | | | |-TableScanIterator::Read()
  15. | | | | |-handler::ha_rnd_next()
  16. | | | | | |-ha_innobase::rnd_next()
  17. |
  18. | |-HashJoinIterator::InitProbeIterator()
  19. | | |-TableScanIterator::Init()
  20. | | | |-handler::ha_rnd_init()
  21. | | | | |-ha_innobase::rnd_init()
  22. |
  23. | ###while循环读取数据###
  24. |-m_root_iterator->Read() 所有Iterator递归Read,此处IteratorHashJoinIterator
  25. | |-HashJoinIterator::Read()
  26. | |-HashJoinIterator::ReadRowFromProbeIterator()
  27. | | |-TableScanIterator::Read()
  28. | | | |-handler::ha_rnd_next()
  29. | | | | |-ha_innobase::rnd_next()
  30. |

https://dev.mysql.com/doc/internals/en/select-structure.html