DSL Recursion

Out of date

This guide contains outdated information pertaining to Kubeflow 1.0. This guide needs to be updated for Kubeflow 1.1.

This page describes how to write recursive functions in the domain specific language (DSL) provided by the Kubeflow Pipelines SDK.

Decorate the recursive function with kfp.dsl.graph_component as illustrated below. The decorator does not require any arguments.

Define the function signature as a standard Python function. The input parameters are .

You can pass pipeline/component output to the recursive function and specify the dependencies explicitly with after() function, similar to the ContainerOp. In the example above, the output of op_a defined in the pipeline is passed to the recursive function and the task_factory_c component is specified to depend on the graph_op_a. The recursive function can also be explicitly specified to depend on the ContainerOps. For example, graph_op_a depends on op_b in the pipeline.

Here is another example where the recursive function call is at the end of the function body, similar to loops.

  1. @dsl.graph_component
  2. def graph_component_a(input_x):
  3.   op_a = task_factory_a(input_x)
  4.   op_b = task_factory_b().after(op_a)
  5.   with dsl.Condition(op_b.output == 'value_x'):
  8. @dsl.pipeline(
  9.   name='pipeline',
  10.   description='shows how to use the recursion.'
  12.   op_a = task_factory_a()
  13.   op_b = task_factory_b()
  14.   graph_op_a = graph_component_a(op_a.output)
  15.   graph_op_a.after(op_b)
  16.   task_factory_c(op_a.output).after(graph_op_a)
  • Type checking does not work for the recursive functions. In other words, The type information that is annotated to the recursive function signature will not be checked.
  • Since the output of the recursive functions cannot be dynamically resolved, the downstream ContainerOps cannot access the output from the recursive functions.
  • A known is that the recursion fails to work when there are multiple recursive function calls in the function body.