cushLEPOR: customising hLEPOR metric using Optuna for higher agreement
  with human judgments or pre-trained language model LaBSE
  Human evaluation has always been expensive while researchers struggle to
trust the automatic metrics. To address this, we propose to customise
traditional metrics by taking advantages of the pre-trained language models
(PLMs) and the limited available human labelled scores. We first re-introduce
the hLEPOR metric factors, followed by the Python version we developed (ported)
which achieved the automatic tuning of the weighting parameters in hLEPOR
metric. Then we present the customised hLEPOR (cushLEPOR) which uses Optuna
hyper-parameter optimisation framework to fine-tune hLEPOR weighting parameters
towards better agreement to pre-trained language models (using LaBSE) regarding
the exact MT language pairs that cushLEPOR is deployed to. We also optimise
cushLEPOR towards professional human evaluation data based on MQM and pSQM
framework on English-German and Chinese-English language pairs. The
experimental investigations show cushLEPOR boosts hLEPOR performances towards
better agreements to PLMs like LaBSE with much lower cost, and better
agreements to human evaluations including MQM and pSQM scores, and yields much
better performances than BLEU (data available at
\url{https://github.com/poethan/cushLEPOR}). Official results show that our
submissions win three language pairs including \textbf{English-German} and
\textbf{Chinese-English} on \textit{News} domain via cushLEPOR(LM) and
\textbf{English-Russian} on \textit{TED} domain via hLEPOR.
