Custom Loss for Pretrained TF HF Models
Maybe a bit of a dumb question but I’m trying to fine tune a simple POS tagger like so:model = transformers.TFAutoModelForTokenClassification.from_pretrained('distilbert-base-multilingual-cased', num_labels=len(LABELS))
    optimizer = tf.keras.optimizers.Adam(learning_rate=lr)
    loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    model.compile(optimizer=optimizer,
                  loss={"loss": loss},
                  metrics=tf.keras.metrics.SparseCategoricalAccuracy())
    model.fit(train,
              epochs=args.epochs)but it complains that the target for getting the gradient isNone:TypeError: Target should be a list or nested structure of Tensors or Variables to be differentiated, but recieved NoneI tried passing the loss directly and as a dictionary like in the snippet here and neither way works. How do I pass it a custom loss properly?I’m at a loss since theexample codepretty much does the same thing I’m doing.