vision.learner

is the module that implements custom Interpretation classes for different vision tasks by inheriting from it.

Interpretation methods for segmenatation models.

top_losses[source][test]

top_losses(sizes:Tuple, k:int=None, largest=True) No tests found for top_losses. To contribute a test please refer to and this discussion.

Reduce flatten loss to give a single loss value for each image

_interp_show[source][test]

Show ImageSegment with color mapping labels

show_xyz[source][test]

show_xyz(i, classes:list=None, sz=10) No tests found for show_xyz. To contribute a test please refer to and this discussion.

show (image, true and pred) from self.ds with color mappings, optionally only plot

_generate_confusion[source][test]

Average and Per Image Confusion: intersection of pixels given a true label, true label sums to 1

_plot_intersect_cm[source][test]

Plot confusion matrices: self.mean_cm or self.single_img_cm generated by _generate_confusion

Let’s show how can be used once we train a segmentation model.

codes = np.loadtxt(camvid/'codes.txt', dtype=str)
get_y_fn = lambda x: path_lbl/f'{x.stem}_P{x.suffix}'

        .split_by_rand_pct()
        .label_from_func(get_y_fn, classes=codes)
        .transform(get_transforms(), tfm_y=True, size=128)
        .databunch(bs=16, path=camvid)
        .normalize(imagenet_stats))

data.show_batch(rows=2, figsize=(7,5))

learn.fit_one_cycle(3,1e-2)
learn.save('mini_train')

Warning: Following results will not make much sense with this underperforming model but functionality will be explained with ease

interpret

Since FlattenedLoss of CrossEntropyLoss() is used we reshape and then take the mean of pixel losses per image. In order to do so we need to pass sizes:tuple to top_losses()

top_losses, top_idxs = interp.top_losses(sizes=(128,128))

(tensor([3.3195, 3.1692, 2.6574, 2.5976, 2.4910, 2.3759, 2.3710, 2.2064, 2.0871,
         2.0834, 2.0479, 1.8645, 1.8412, 1.7956, 1.7013, 1.6126, 1.6015, 1.5470,
 tensor([12,  4, 17, 13, 19, 18,  7,  8, 10,  1, 15,  0,  2,  9, 16, 11, 14,  5,
          6,  3]))

Next, we can generate a confusion matrix similar to what we usually have for classification. Two confusion matrices are generated: mean_cm which represents the global label performance and single_img_cm which represents the same thing but for each individual image in dataset.

Values in the matrix are calculated as:

begin{align} CM_{ij} & = IOU(Predicted , True | True) end{align}

Or in plain english: ratio of pixels of predicted label given the true pixels

learn.data.classes

array(['Animal', 'Archway', 'Bicyclist', 'Bridge', 'Building', 'Car', 'CartLuggagePram', 'Child', 'Column_Pole',
       'Fence', 'LaneMkgsDriv', 'LaneMkgsNonDriv', 'Misc_Text', 'MotorcycleScooter', 'OtherMoving', 'ParkingBlock',
       'Pedestrian', 'Road', 'RoadShoulder', 'Sidewalk', 'SignSymbol', 'Sky', 'SUVPickupTruck', 'TrafficCone',
       'TrafficLight', 'Train', 'Tree', 'Truck_Bus', 'Tunnel', 'VegetationMisc', 'Void', 'Wall'], dtype='<U17')

((32, 32), (20, 32, 32))

_plot_intersect_cm first displays a dataframe showing per class score using the IOU definition we made earlier. These are the diagonal values from the confusion matrix which is displayed after.

df = interp._plot_intersect_cm(mean_cm, "Mean of Ratio of Intersection given True Label")

Next let’s look at the single worst prediction in our dataset. It looks like this dummy model just predicts everything as Road :)

i = top_idxs[0]
df = interp._plot_intersect_cm(single_img_cm[i], f"Ratio of Intersection given True Label, Image:{i}")

Finally we will visually inspect this single prediction

interp.show_xyz(i, sz=15)

Warning: With matplotlib colormaps the max number of unique qualitative colors is 20. So if len(classes) > 20 then close class indexes may be plotted with the same color. Let’s fix this together :)

Interpretation methods for classification models.

Warning: ObjectDetectionInterpretation is not implemented yet. Feel free to implement it :)

©2021 fast.ai. All rights reserved.
Site last generated: Jan 5, 2021