Sending System and User Prompt in Deployed endpoint
I just deployeddolphin2.2-mistralusing hugging face DEPLOY->INFERENCE ENDPOINT and got it working.It does take inputs as payload and working fine. My issue is, what value should I send to have GPT like experience? Something like system, user and assistant.How can I change the system prompt, and send chain of conversational messages that I can do like that of Openai gpt?Payload like :{“inputs”: “What are different ways to copy with long stressful day?”,“parameters”: {“max_new_tokens”: 100}}works.But, how can I send conversational type messages?Also, if I deploy a model, how can I change the code myself?Sorry if this is 2 questions merged into one.
2023-11-16T12:56:37Z
Lookup how to do custom endpoint handler which may help your use case. You can send whatever you need and then in the endpoint handler you process the input and pass it to the pipeline.huggingface.coCreate custom Inference HandlerWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2023-11-17T16:39:18Z
Do you have example of payload? I am pretty new and I am sure I am going to spend a lot of time on this which I am not sure I would be able to.
2023-11-17T16:58:04Z
in the page I linked to  above has quite a few examples listed in the linked repositories. Are you looking for something different?