Configuration Settings and Compiling Modes

    As a rule, the attributes in the config module should not be modified inside the user code.

    Theano’s code comes with default values for these attributes, but you canoverride them from your .theanorc file, and override those values in turn bythe THEANO_FLAGS environment variable.

    The order of precedence is:

    • an assignment in
    • an assignment in the .theanorc file (or the file indicated in THEANORC) You can display the current/effective configuration at any time by printingtheano.config. For example, to see a list of all active configurationvariables, type this from the command-line:

    For more detail, see in the library.

    Exercise

    Consider the logistic regression:

    Modify and execute this example to run on CPU (the default) with floatX=float32 andtime the execution using the command line time python file.py. Save your codeas it will be useful later on.

    Note

    • Apply the Theano flag floatX=float32 (through theano.config.floatX) in your code.
    • Cast inputs before storing them into a shared variable.
    • Circumvent the automatic cast of int32 with float32 to float64:
      • Insert manual cast in your code or use [u]int{8,16}.
      • Note that a new casting mechanism is being developed.

    Every time theano.function is called,the symbolic relationships between the input and output Theano _variables_are optimized and compiled. The way this compilation occursis controlled by the value of the mode parameter.

    • 'FAST_COMPILE': Apply just a few graph optimizations and only use Python implementations. So GPU is disabled.

    • : Apply all optimizations and use C implementations where possible.

      • 'DebugMode': Verify the correctness of all optimizations, and compare C and Python
      • implementations. This mode can take much longer than the other modes, but can identifyseveral kinds of problems.

    The default mode is typically FAST_RUN, but it can be controlled viathe configuration variable config.mode,which can be overridden by passing the keyword argument totheano.function.

    Note

    For debugging purpose, there also exists a MonitorMode (which has noshort name). It can be used to step through the execution of a function:see for details.

    Linkers

    A mode is composed of 2 things: an optimizer and a linker. Some modes,like NanGuardMode and , add logic around theoptimizer and linker. DebugMode uses its own linker.

    You can select which linker to use with the Theano flag .Here is a table to compare the different linkers.

    For more detail, see Mode in the library.

    The optimizers Theano provides are summarized below to indicate the trade-offsone might make between compilation time and execution time.

    These optimizers can be enabled globally with the Theano flag: optimizer=nameor per call to theano functions with theano.function(…mode=theano.Mode(optimizer="name")).

    For a detailed list of the specific optimizations applied for each of theseoptimizers, see . Also, see Unsafe optimization and for other trade-off.

    Using DebugMode

    While normally you should use the FAST_RUN or FAST_COMPILE mode,it is useful at first (especially when you are defining new kinds ofexpressions or new optimizations) to run your code using the DebugMode(available via mode='DebugMode). The DebugMode is designed torun several self-checks and assertions that can help diagnosepossible programming errors leading to incorrect output. Note thatDebugMode is much slower than FAST_RUN or FAST_COMPILE souse it only during development (not when you launch 1000 processes on acluster!).

    DebugMode is used as follows:

    If any problem is detected, DebugMode will raise an exception according towhat went wrong, either at call time (f(5)) or compile time (f = theano.function(x, 10 * x, mode='DebugMode')). These exceptionsshould not be ignored; talk to your local Theano guru or email theusers list if you cannot make the exception go away.

    Some kinds of errors can only be detected for certain input value combinations.In the example above, there is no way to guarantee that a future call to, sayf(-1), won’t cause a problem. DebugMode is not a silver bullet.

    If you instantiate DebugMode using the constructor (see DebugMode)rather than the keyword DebugMode you can configure its behaviour viaconstructor arguments. The keyword version of DebugMode (which you get by using )is quite strict.

    For more detail, see in the library.