Model assets for the first Mixture-of-Lora technique applied to Llama. https://bit.ly/48bqshl
crumb
crumb
machine-learnoooooooor's profile picture
charlesdedampierre's profile picture
nisten's profile picture
·
AI & ML interests
None yet
Recent Activity
updated
a Space
3 days ago
crumbs-playground/README
updated
a collection
13 days ago
CLM_R
updated
a dataset
17 days ago
crumbs-playground/loss-balancing-clmr-8B-lora-rb256-cb256-bs8-lr1e-05-gn0.5-eb0.01
Organizations
GPT2 Models using Linear layers instead of Conv layers for convenience.
Smaller models trained locally on my 2xA6000 Lambda Vector
Model assets for the first Mixture-of-Lora technique applied to Llama. https://bit.ly/48bqshl
First Prototype of the second iteration of MoLora utilizing mixture of expert techniques applied to the Llama2 model.
GPT2 Models using Linear layers instead of Conv layers for convenience.
Parts of Meta's LlamaV2 models, chopped up and trained. CoreX means the first X layers were kept.
Smaller models trained locally on my 2xA6000 Lambda Vector