Sentence Similarity for Code Generation related tasks
I am working on code generation using OpenAI Codex.Specifically, I’m making a personal project that lets users enter a specific “task” that they want to do related to code generation.Currently I have around 10 tasks e.g. explain a piece of code, convert some code to a one liner, write unit-tests for a function.My goal is to recognize the user’s intent. Normally, the approach would be to use a multi-class classification model but due to some time & data constraints, I want to use sentence similarity instead.Basically my idea is that for each task:Generate a list of of the 3-4 most common prompts that users can enter for this taskFor any user queryPerform a sentence similarity searchFor the task with the top similarity scoreIf score > thresholdrecognize this task as the user’s actual intentElse user intent is not among current set of tasksMy questions are:Is this approach any good?Should I use a Passage Ranking or Semantic Textual Similarity model? Or something else?Is there any relevant dataset on which I can finetune the model to work on prompts specific to code generation?
2022-03-28T07:34:46Z
Hey there! There is this blog post from@espejelomarusing Sentence Transformers by@nreimersthat will answer some of your questionsSentence Transformers for semantic search - a Hugging Face Space by sentence-transformers. I hope this helps