Lily-Cybersecurity-7B-Uncensored-GGUF_Q5_K_M

license: apache-2.0 library_name: llama.cpp pipeline_tag: text-generation tags:

gguf
quantized
cybersecurity
uncensored
q4_k_m
q5_k_m base_model: NiroshanDb23/Lily-Cybersecurity-7B-Uncensored-GGUF

Lily-Cybersecurity-7B-Uncensored-GGUF_Q5_K_M

Credits & License

This is a re-quantized version of an existing GGUF model. (And size of it converted from 13.5GB to 4.07GB) So it can be used on CPU with 16GB of RAM (Also can be run on Google Colab).

Lineage:

Original Model: segolilylabs/Lily-Cybersecurity-7B-v0.2 (License: Apache 2.0)
Intermediate GGUF (source for re-quantization): NiroshanDb23/Lily-Cybersecurity-7B-Uncensored-GGUF (quantized_by: NiroshanDH, includes abliteration modifications)

My Contribution:

Re-quantized the intermediate GGUF file into Q4_K_M and Q5_K_M formats to reduce RAM usage for inference.
Alarm: Quantized version can hallucinate.
Recommended context window: 4096 tokens (or higher if hardware permits). License: This derivative work is distributed under the Apache 2.0 license, inherited from the original model. Please refer to the original model card for full terms and disclaimer.

Donation address (if you want to see more quantized uncensored models - or in future create of more uncensored model): 1Kk32KvJVjcyTMZ44eos4eZtme1quTqFAg (BTC)

Credits & License

This is a re-quantized version of an existing GGUF model. (And size of it converted from 13.5GB to 4.07GB) So it can be used on CPU with 16GB of RAM (Also can be run on Google Colab).

Lineage:

Original Model: segolilylabs/Lily-Cybersecurity-7B-v0.2 (License: Apache 2.0)

Intermediate GGUF (source for re-quantization): NiroshanDb23/Lily-Cybersecurity-7B-Uncensored-GGUF (quantized_by: NiroshanDH, includes abliteration modifications)

My Contribution:

Re-quantized the intermediate GGUF file into Q4_K_M and Q5_K_M formats to reduce RAM usage for inference.

Alarm: Quantized version can hallucinate.

Recommended context window: 4096 tokens (or higher if hardware permits). License: This derivative work is distributed under the Apache 2.0 license, inherited from the original model. Please refer to the original model card for full terms and disclaimer.

Donation address (if you want to see more quantized uncensored models - or in future create of more uncensored model): 1Kk32KvJVjcyTMZ44eos4eZtme1quTqFAg (BTC)