Custom domain specific training text-to-text
Hell Hugginface exerts,I am trying to solve a very specific domain problem. I have a project where I must take input text that contains domain specific codes in the text and translate it to a specific output.I am usingABCandFOOto just convey the basic idea here, because I do not want to distract with problem statement.  However, my domain iselectrical engineeringdata.  There will be words like:Port,VSS,GND,CHA, etc.  I have a PDF document specification that explains these words, do you think I need to throw this into the fine-tuning flow?In this example, I have an input text, and what I am showing you is the output result that I am trying to produce from the model. This example showsinput text→output result I want, for example:! Port[1] = ABCZ4_FOO1-model->ABC_c_FOO1or! Port[4] = ABC4_FOO1-model->ABC_t_FOO1There are many variations of this input text, where in some cases it would beABC3instead ofABCZ4, so in this case the input/output would be:! Port[1] = ABC3_FOO1-model->ABC_c_FOO1What I am trying to figure out is what would be the best model transformer, base mode model and then training task.  Should I use translation, masking, or something else?You can see that the input has individual character input tokens, and the sequence of characters gives more context.  My instincts is that atransformermakes sense here because “attention” is needed on the specific character tokens to betranslatedto an output.Here is a snapshot of a completed table of input/output. You could imagine that I am going to use this data to train the model. There is a ton of variations of possibilities of the input tokens that could be provided with different text.  BTW, I have about 700 training input/output samples.  Do you think this would be enough or would I need generate more?image259×621 5.21 KBThanks!
2023-03-14T19:09:59Z
Just to start, are you positive that the problem can’t be solved with traditional code (e.g., using regexes)? The examples you posted look like they have a sufficiently simple pattern that AI isn’t needed.
2023-03-14T23:15:33Z
Thanks@dblakely,Yes I’m sure we will not be able to edit regent. There are so many variations that users can input that I could not put it all in the example. It is something we will really have to develop a model for. Do you have any ideas?
2023-06-15T22:28:00Z
Just bumping this thread, I’m curious of the possible solutions to this.