Overview of the compilation pipeline

By creating Theano Variables using or theano.tensor.dmatrix or by usingTheano functions such as theano.tensor.sin ortheano.tensor.log, the user builds a computation graph. Thestructure of that graph and details about its components can be foundin the article.

Compilation of the computation graph

Once the user has built a computation graph, she can usetheano.function in order to make one or more functions thatoperate on real data. function takes a list of input as well as a list of output Variables that define aprecise subgraph corresponding to the function(s) we want to define,compile that subgraph and produce a callable.

Here is an overview of the various steps that are done with thecomputation graph in the compilation phase:

The subgraph given by the end user is wrapped in a structure calledFunctionGraph. That structure defines several hooks on adding andremoving (pruning) nodes as well as on modifying links between nodes(for example, modifying an input of an Apply node) (see thearticle about for more information).

Some relevant Features are typically added to theFunctionGraph, namely to prevent any optimization from operating inplace oninputs declared as immutable.

Once the FunctionGraph is made, an is produced bythe mode passed to function (the Mode basically has twoimportant fields, linker and optimizer). That optimizer isapplied on the FunctionGraph using its optimize() method.

The optimizer is typically obtained through .

Once the computation graph is optimized, the isextracted from the Mode. It is then called with the FunctionGraph asargument toproduce a thunk, which is a function with no arguments thatreturns nothing. Along with the thunk, one list of input containers (atheano.gof.Container is a sort of object that wraps another and doestype casting) and one list of output containers are produced,corresponding to the input and output Variables as well as the updatesdefined for the inputs when applicable. To perform the computations,the inputs must be placed in the input containers, the thunk must becalled, and the outputs must be retrieved from the output containerswhere the thunk put them.

The linker is where some options take effect: the strict flag ofan input makes the associated input container do type checking. Theborrow flag of an output, if False, adds the output to ano_recycling list, meaning that when the thunk is called theoutput containers will be cleared (if they stay there, as would be thecase if borrow was True, the thunk would be allowed to reuse (or“recycle”) the storage).

Note

Compiled libraries are stored within a specific compilation directory,which by default is set to $HOME/.theano/compiledir_xxx, where identifies the platform (under Windows the default locationis instead $LOCALAPPDATA\Theano\compiledir_xxx). It may be manually setto a different location either by setting config.compiledir or, either within your Python script or byusing one of the configuration mechanisms described in config.

The compile cache is based upon the C++ code of the graph to be compiled.So, if you change compilation configuration variables, such asconfig.blas.ldflags, you will need to manually remove your compile cache,using Theano/bin/theano-cache clear

The thunk returned by the linker along with input and outputcontainers is unwieldy. hides that complexity away sothat it can be used like a normal function with arguments and returnvalues.