Tabular data

Helper functions to get data in a DataLoaders in the tabular application and higher class TabularDataLoaders

The main class to get your data ready for model training is TabularDataLoaders and its factory methods. Checkout the for examples of use.

This class should not be used directly, one of the factory methods should be preferred instead. All those factory methods accept as arguments:

• cat_names: the names of the categorical variables
• cont_names: the names of the continuous variables
• y_names: the names of the dependent variables
• y_block: the TransformBlock to use for the target
• bs: the batch size
• val_bs: the batch size for the validation (defaults to bs)
• shuffle_train: if we shuffle the training DataLoader or not
• n: overrides the numbers of elements in the dataset
• device: the PyTorch device to use (defaults to default_device())

TabularDataLoaders.from_df[source]

Create from df in path using procs

path = untar_data(URLs.ADULT_SAMPLE)
df = pd.read_csv(path/'adult.csv', skipinitialspace=True)

dls = TabularDataLoaders.from_df(df, path, procs=procs, cat_names=cat_names, cont_names=cont_names, 
                                 y_names="salary", valid_idx=list(range(800,1000)), bs=64)

TabularDataLoaders.from_csv[source]

Create from csv file in path using procs

cat_names = ['workclass', 'education', 'marital-status', 'occupation', 'relationship', 'race']
cont_names = ['age', 'fnlwgt', 'education-num']
procs = [Categorify, FillMissing, Normalize]
dls = TabularDataLoaders.from_csv(path/'adult.csv', path=path, procs=procs, cat_names=cat_names, cont_names=cont_names, 
                                  y_names="salary", valid_idx=list(range(800,1000)), bs=64)

External structured data files can contain unexpected spaces, e.g. after a comma. We can see that in the first row of adult.csv "49, Private,101320, ...". Often trimming is needed. Pandas has a convenient parameter skipinitialspace that is exposed by ). Otherwise category labels use for inference later such as workclass:Private will be categorized wrongly to 0 or "#na#" if training label was read as " Private". Let’s test this feature.