KeyError in vocab if the vocab is built on a subset of the initially read data

[zeeraktalat]

So I'm getting a strange issue, where I'm trying to read in a dataset (from a single file), split it into a train, dev, and test set. If I read it in using TabularDataset, then split the data, and train the vocab on the first split, I get KeyErrors, however if I split the dataset files prior to reading it in, no such errors occur.

Dataset I've been running into this issue: https://github.com/t-davidson/hate-speech-and-offensive-language/tree/master/data

To Reproduce

1. Read in data
2. Split the data (using tabulardataset.split) into n sets
3. Build your vocab on the training set
4. Iterate over dev/test set

• PyTorch Version (e.g., 1.0): 1.20
• OS (e.g., Linux): OSX
• How you installed PyTorch (conda, pip, source): pip
• Python version: 3.7

[zhangguanheng66]

@mttk Are we supposed to read and split a single file for train/valid/test datasets.

[mttk]

@ZeerakW could you paste the code you're using for both loads? I'm interested in the Field config

[zeeraktalat]

Sure, just for completeness here is everything. I'm just heading out now, but will post a notebook later.

import os
import sys
import torch.nn as nn
import torch.optim as optim
from torchtext.data import TabularDataset, BucketIterator, Field

text_label = Field(sequential = True,
                   include_lengths = False,
                   use_vocab = True,
                   pad_token = "<PAD>",
                   unk_token = "<UNK>")

int_label = Field(sequential = False,
                  include_lengths = False,
                  use_vocab = False,
                  pad_token = None,
                  unk_token = None)

device = 'cpu'
data_dir = 'data/'
# data_file = 'davidson_offensive.csv'
data_file = 'davidson_test.csv'
path = os.path.join(data_dir, data_file)
file_format = 'csv'

text_field = text_label
label_field = int_label

fields = [('', None), ('CF_count', None), ('hate_speech', None), ('offensive', None), ('neither', None),
          ('label', label_field), ('text', text_field)]

data = TabularDataset(path, format = file_format, fields = fields, skip_header = True)
train_data, test = data.split(split_ratio = 0.8, stratified = True)
loaded = (train_data, test)

And here the loads diverge for one line:

Working (train vocab on entire dataset)

text_field.build_vocab(data)

Not working

text_field.build_vocab(train_data)

Reconverging:

VOCAB_SIZE = len(text_field.vocab)
batch_sizes = (32, 32)
tmp_train, tmp_test = BucketIterator.splits(loaded, batch_sizes = batch_sizes, sort_key = lambda x: len(x.text), device = device, shuffle = True, repeat = False)

and then (first line works, second fails).

next(iter(tmp_train))
next(iter(tmp_test))

[zeeraktalat]

Here's a gist: https://gist.github.com/ZeerakW/b9da3c9a9c456d404b71cc7f82429dc2