training

    The fastai library structures its training process around the Learner class, whose object binds together a PyTorch model, a dataset, an optimizer, and a loss function; the entire learner object then will allow us to launch training.

    defines this Learner class, along with the wrapper around the PyTorch optimizer that the library uses. It defines the basic training loop that is used each time you call the method (or one of its variants) in fastai. This training loop is very bare-bones and has very few lines of codes; you can customize it by supplying an optional Callback argument to the method.

    callback defines the class and the CallbackHandler class that is responsible for the communication between the training loop and the ‘s methods. The CallbackHandler maintains a state dictionary able to provide each object all the information of the training loop it belongs to, putting any imaginable tweaks of the training loop within your reach.

    callbacks implements each predefined class of the fastai library in a separate module. Some modules deal with scheduling the hyperparameters, like callbacks.one_cycle, and callback.general_sched. Others allow special kinds of training like (mixed precision) and callbacks.rnn. The and callbacks.hooks are useful to save some internal data generated in the training loop.

    then uses these callbacks to implement useful helper functions. Lastly, metrics contains all the functions and classes you might want to use to evaluate your training results; simpler metrics are implemented as functions while more complicated ones as subclasses of . For more details on implementing metrics as Callback, please refer to .

    We’ll do a quick overview of the key pieces of fastai’s training modules. See the separate module docs for details on each.

    Import required modules and prepare :

    URLs.MNIST_SAMPLE is a small subset of the classic MNIST dataset containing the images of just 3’s and 7’s for the purpose of demo and documentation here. Common datasets can be downloaded with - which we will use to create an ImageDataBunch object

    Basic training with Learner

    1. model = simple_cnn((3,16,16,2))

    The class plays a central role in training models; when you create a Learner you need to specify at the very minimum the and model to use.

    1. learn = Learner(data, model)

    These are enough to create a Learner object and then use it to train a model using its method. If you have a CUDA-enabled GPU, it will be used automatically. To call the method, you have to at least specify how many epochs to train for.

    Total time: 00:03

    To see how our training is going, we can request that it reports various kinds of after each epoch. You can pass it to the constructor, or set it later. Note that metrics are always calculated on the validation set.

    1. learn.metrics=[accuracy]
    2. learn.fit(1)

    Total time: 00:02

    Extending training with callbacks

    You can use s to modify training in almost any way you can imagine. For instance, we’ve provided a callback to implement Leslie Smith’s 1cycle training method.

    1. learn.fit(1, callbacks=cb)

    Total time: 00:02

    The Recorder callback is automatically added for you, and you can use it to see what happened in your training, e.g.:

    Many of the callbacks can be used more easily by taking advantage of the extensions in train. For instance, instead of creating OneCycleScheduler manually as above, you can simply call :

    1. learn.fit_one_cycle(1)

    Total time: 00:03

    Applications

    Note that if you’re training a model for one of our supported applications, there’s a lot of help available to you in the application modules:

    For instance, let’s use (from vision) to quickly fine-tune a pre-trained Imagenet model for MNIST (not a very practical approach, of course, since MNIST is handwriting and our model is pre-trained on photos!).

    Total time: 02:06

    Open in GCP Notebooks

    Company logo