Loss function used in run_mmimdb.py
Hi,Why BCE was used ingithub.comhuggingface/transformers/blob/8e74eca7f2b3235f8d5340d66361ea656c67bac7/examples/research_projects/mm-imdb/run_mmimdb.py#L524logger.info("Training/evaluation parameters %s", args)# Trainingif args.do_train:train_dataset = load_examples(args, tokenizer, evaluate=False)label_frequences = train_dataset.get_label_frequencies()label_frequences = [label_frequences[l] for l in labels]label_weights = (torch.tensor(label_frequences, device=args.device, dtype=torch.float) / len(train_dataset)) ** -1criterion = nn.BCEWithLogitsLoss(pos_weight=label_weights)global_step, tr_loss = train(args, train_dataset, model, tokenizer, criterion)logger.info(" global_step = %s, average loss = %s", global_step, tr_loss)# Saving best-practices: if you use defaults names for the model, you can reload it using from_pretrained()if args.do_train and (args.local_rank == -1 or torch.distributed.get_rank() == 0):logger.info("Saving model checkpoint to %s", args.output_dir)# Save a trained model, configuration and tokenizer using `save_pretrained()`.# They can then be reloaded using `from_pretrained()`model_to_save = (model.module if hasattr(model, "module") else modelThe labels seem not binary to me.label_frquencieslooks likeCounter({'Drama': 8415, 'Comedy': 5107, 'Romance': 3226, 'Thriller': 3110, 'Crime': 2287, 'Action': 2154, 'Adventure': 1609, 'Horror': 1603, 'Mystery': 1231, 'Sci-Fi': 1212, 'Documentary': 1195, 'Fantasy': 1162, 'Family': 975, 'War': 804, 'Biography': 772, 'History': 663, 'Music': 632, 'Animation': 586, 'Musical': 503, 'Western': 423, 'Sport': 379, 'Short': 281, 'Film-Noir': 202})What may I miss? Thanks.