inference: false library_name: transformers language:

en
fr
de
es
it
pt
ja
ko
zh
ar
el
fa
pl
id
cs
he
hi
nl
ro
ru
tr
uk
vi license: cc-by-nc-4.0 extra_gated_prompt: By submitting this form, you agree to the License Agreement and acknowledge that the information you provide will be collected, used, and shared in accordance with Cohere’s Privacy Policy. You’ll receive email updates about C4AI and Cohere research, events, products and services. You can unsubscribe at any time. extra_gated_fields: Name: text Affiliation: text Country: country I agree to use this model for non-commercial use ONLY: checkbox tags:
abliterated
uncensored
mlx
mlx-my-repo base_model: huihui-ai/aya-expanse-32b-abliterated

KYUNGYONG/aya-expanse-32b-abliterated-mlx-3Bit

The Model KYUNGYONG/aya-expanse-32b-abliterated-mlx-3Bit was converted to MLX format from huihui-ai/aya-expanse-32b-abliterated using mlx-lm version 0.22.1.

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("KYUNGYONG/aya-expanse-32b-abliterated-mlx-3Bit")

prompt="hello"

if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate model, tokenizer = load("KYUNGYONG/aya-expanse-32b-abliterated-mlx-3Bit") prompt="hello" if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None: messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) response = generate(model, tokenizer, prompt=prompt, verbose=True)