玩模型开放模型 · 清晰来源 · 自主选择
返回模型目录
huihui-ai /

Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF

HF 源站

huihui-ai/Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF is an uncensored open-source model hosted on Hugging Face.

llama.cppGGUFLicense: mittransformersabliterateduncensoredquantizeddeepseekdeepseek-v4deepseek-v4-flashmoemixture-of-experts2-bit4-bitiq2_xxsq2_kq4_kds4apple-siliconmetalconversational
README.md说明 / 使用方法

license: mit library_name: transformers base_model:

  • deepseek-ai/DeepSeek-V4-Flash base_model_relation: quantized quantized_by:
  • antirez
  • huihui.ai tags:
  • abliterated
  • uncensored
  • GGUF
  • quantized
  • deepseek
  • deepseek-v4
  • deepseek-v4-flash
  • moe
  • mixture-of-experts
  • 2-bit
  • 4-bit
  • iq2_xxs
  • q2_k
  • q4_k
  • ds4
  • apple-silicon
  • metal
  • llama.cpp

extra_gated_prompt: >- 使用警告

“**敏感或争议性输出风险**”：该模型的安全过滤机制已大幅削弱，可能会生成敏感、具有争议或不当的内容。用户应保持谨慎并严格审核生成的输出。

“**不适合所有受众**”：由于内容过滤机制有限，该模型的输出可能不适合公共场合、未成年人或对安全性要求较高的应用场景。

“**法律与伦理责任**”：用户必须确保自身使用行为符合当地法律和伦理标准。生成的内容可能带有法律或伦理风险，用户须对一切后果承担全部责任。

“**研究与实验用途**”：建议将此模型用于研究、测试或受控环境中，避免直接用于生产环境或面向公众的商业应用。

“**监控与审核建议**”：强烈建议用户实时监控模型输出，并在必要时进行人工审核，以防止不当内容的传播。

“**不提供默认安全保证**”：与标准模型不同，此模型未经过严格的安全优化。huihui.ai 对因使用该模型而产生的任何后果不承担责任。

huihui-ai/Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF

这是通过 abliteration 技术创建的 deepseek-ai/DeepSeek-V4-Flash 无审查（uncensored）版本。

这些量化版本专为 DS4（antirez/ds4）和 llama.cpp 推理引擎设计。 它们在其他推理引擎上可能会或可能无法正常工作（理论上应该可行，但 MTP 模型除外，它需要特定的加载器）。

注意

  1. Q2 版本存在一定的拒答率。用于编写代码应该没问题，而其他版本仍在测试中。

  2. 请根据您的 GPU 显存大小选择合适的模型。所有模型均可在 Fringe210/llama.cpp-deepseek-v4-flash-cuda（支持多 GPU）和 ds4（支持多 GPU）下运行。

  3. ds4 现已支持多 GPU 运行。关于如何使用的更多信息，请参阅 x.com/support_huihui。

DS4 Unix 域套接字 (UDS) 加速补丁

通过使用 Unix 域套接字（Unix Domain Sockets）替换 TCP 本地回环（loopback），大幅加速同机多 GPU 分层切分推理（协调器 + 工作节点模式）。 开源地址 👉 huihui-support/ds4/tree/uds

DS4 张量并行 (Tensor-Parallel) 加速补丁

使用单进程在单台机器上大幅提升多 GPU 分层切分推理速度，全面支持消费级显卡。 开源地址 👉 huihui-support/ds4/tree/tp

文件列表

模板文件来源于 antirez/deepseek-v4-gguf/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2.gguf。

文件大小路由专家 (ffn_{gate,up,down}_exps)其他部分
Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-Q2.gguf80.8 GiBIQ2_XXS (gate, up) + Q2_K (down)Q8_0 attn proj / shared experts / output, F16 router + embed + indexer + compressor + HC, F32 norms / sinks / bias
Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-IQ2_XXS.gguf74.7 GiBIQ2_XXS (gate, up, down)同上
Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-Q2_K.gguf92.8 GiBQ2_K (gate, up, down)同上
Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-Q4_K.gguf153 GiBQ4_K (gate, up, down)同上
DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf3.6 GiBMTP / 投机解码支持（可选，非独立运行）。

在 128 GB 显存/内存的 Mac 机器上使用 q2，在内存/显存 ≥ 256 GB 的机器上使用 q4，两者均可搭配 MTP 进行可选的投机解码。

下载

hf download huihui-ai/Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF \
  --local-dir ./huihui-ai/Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF \
  --token hf_xxx

llama.cpp

使用 Fringe210/llama.cpp-deepseek-v4-flash-cuda 程序（需要编译 llama-cli）：

llama-cli -m huihui-ai/Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-Q2.gguf -n 40960

DS4

测试环境

Windows, WSL2, Ubuntu 24.04, RTX 6000 Pro (96GB), CUDA 13.0
在此环境下，推理速度可达到每秒 35 个 token 以上。
尚未在 Apple 环境下进行测试。

支持的硬件

仅测试了 RTX 6000 Pro；其他硬件尚未进行测试。

Metal：配备 96GB 内存的 MacBook、Mac Studio 级别的机器。

NVIDIA CUDA：DGX Spark、RTX 6000 Pro。

安装

git clone https://github.com/antirez/ds4
cd ds4

make

CLI 命令行使用

export CUDA_VISIBLE_DEVICES=0

./ds4 -m ./huihui-ai/Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF/Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-Q2.gguf \
  -p "Explain Redis streams in one paragraph."

服务端运行

export CUDA_VISIBLE_DEVICES=0
./ds4-server \
  --cuda \
  -m ././huihui-ai/Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF/Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-Q2.gguf \
  --ctx 131072 \
  --kv-disk-dir ./ds4-kv-cache \
  --kv-disk-space-mb 32768 \
  --power 75 \
  --warm-weights

curl 测试

curl http://127.0.0.1:8000/v1/models

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [
      {"role": "user", "content": "hello"}
    ],
    "temperature": 0.7,
    "max_tokens": 512,
    "stream": false
  }'

许可证

MIT。基础模型版权归 DeepSeek 所有；GGUF 文件根据基础模型的发布条款重新分发。

使用警告

  • 敏感或争议性输出风险：该模型的安全过滤机制已大幅削弱，可能会生成敏感、具有争议或不当的内容。用户应保持谨慎并严格审核生成的输出。

  • 不适合所有受众：由于内容过滤机制有限，该模型的输出可能不适合公共场合、未成年人或对安全性要求较高的应用场景。

  • 法律与伦理责任：用户必须确保自身使用行为符合当地法律和伦理标准。生成的内容可能带有法律或伦理风险，用户须对一切后果承担全部责任。

  • 研究与实验用途：建议将此模型用于研究、测试或受控环境中，避免直接用于生产环境或面向公众的商业应用。

  • 监控与审核建议：强烈建议用户实时监控模型输出，并在必要时进行人工审核，以防止不当内容的传播。

  • 不提供默认安全保证：与标准模型不同，此模型未经过严格的安全优化。huihui.ai 对因使用该模型而产生的任何后果不承担责任。

赞助支持

如果您喜欢这个模型，请点赞并关注我们以获取更多更新。
您可以关注 x.com/support_huihui 获取 huihui.ai 的最新模型动态。

您的支持能帮助我们继续进一步开发与改进，一杯咖啡足矣。

  • 比特币（BTC）：
  bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70ge
  • 在 Ko-fi (https://ko-fi.com/huihuiai) 上支持我们的工作！
源站原文

license: mit library_name: transformers base_model:

  • deepseek-ai/DeepSeek-V4-Flash base_model_relation: quantized quantized_by:
  • antirez
  • huihui.ai tags:
  • abliterated
  • uncensored
  • GGUF
  • quantized
  • deepseek
  • deepseek-v4
  • deepseek-v4-flash
  • moe
  • mixture-of-experts
  • 2-bit
  • 4-bit
  • iq2_xxs
  • q2_k
  • q4_k
  • ds4
  • apple-silicon
  • metal
  • llama.cpp

extra_gated_prompt: >- Usage Warnings

“**Risk of Sensitive or Controversial Outputs**“: This model’s safety filtering has been significantly reduced, potentially generating sensitive, controversial, or inappropriate content. Users should exercise caution and rigorously review generated outputs.

“**Not Suitable for All Audiences**:“ Due to limited content filtering, the model’s outputs may be inappropriate for public settings, underage users, or applications requiring high security.

“**Legal and Ethical Responsibilities**“: Users must ensure their usage complies with local laws and ethical standards. Generated content may carry legal or ethical risks, and users are solely responsible for any consequences.

“**Research and Experimental Use**“: It is recommended to use this model for research, testing, or controlled environments, avoiding direct use in production or public-facing commercial applications.

“**Monitoring and Review Recommendations**“: Users are strongly advised to monitor model outputs in real-time and conduct manual reviews when necessary to prevent the dissemination of inappropriate content.

“**No Default Safety Guarantees**“: Unlike standard models, this model has not undergone rigorous safety optimization. huihui.ai bears no responsibility for any consequences arising from its use.

huihui-ai/Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF

This is an uncensored version of deepseek-ai/DeepSeek-V4-Flash created with abliteration.

This quants are specific for the DS4(antirez/ds4) and llama.cpp inference engine. They may work with other inference engines or not (they should, but not the MTP model which requires a specific loader).

Note

  1. The Q2 version has a certain refusal rate. It should be fine for writing code, while the other versions are still under testing.

  2. Choose the appropriate model based on the size of your GPU. All models can run under both Fringe210/llama.cpp-deepseek-v4-flash-cuda(supports multi-GPU) and ds4(supports multi-GPU).

  3. ds4 now supports multi-GPU operation. For more information on how to use it, please refer to x.com/support_huihui

DS4 Unix Domain Socket (UDS) Acceleration Patch

Dramatically accelerate multi-GPU layer-splitting inference on the same machine (coordinator + worker mode) by replacing TCP loopback with Unix Domain Sockets. open source 👉 huihui-support/ds4/tree/uds

DS4 Tensor-Parallel Acceleration Patch

Dramatically speed up multi-GPU layer-splitting inference on a single machine using a single process, with full support for consumer-grade graphics cards. open source 👉 huihui-support/ds4/tree/tp

Files

The Template FILE comes from antirez/deepseek-v4-gguf/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2.gguf.

FileSizeRouted experts (ffn_{gate,up,down}_exps)Everything else
Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-Q2.gguf80.8 GiBIQ2_XXS (gate, up) + Q2_K (down)Q8_0 attn proj / shared experts / output, F16 router + embed + indexer + compressor + HC, F32 norms / sinks / bias
Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-IQ2_XXS.gguf74.7 GiBIQ2_XXS (gate, up, down)same as above
Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-Q2_K.gguf92.8 GiBQ2_K (gate, up, down)same as above
Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-Q4_K.gguf153 GiBQ4_K (gate, up, down)same as above
DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf3.6 GiBMTP / speculative-decoding support (optional, not standalone).

Use q2 on 128 GB Mac machines, q4 on machines with ≥ 256 GB RAM, pair either with MTP for optional speculative decoding.

Download

hf download huihui-ai/Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF \
  --local-dir ./huihui-ai/Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF \
  --token hf_xxx

llama.cpp

Use the Fringe210/llama.cpp-deepseek-v4-flash-cuda program (llama-cli needs to be compiled)

llama-cli -m huihui-ai/Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-Q2.gguf -n 40960

DS4

Test environment

Windows, WSL2, Ubuntu 24.04, RTX 6000 Pro (96GB), CUDA 13.0
In this environment, inference can reach more than 35 tokens per second.
Not tested in the Apple environment.

Supported Hardware

Only the RTX 6000 Pro has been tested; other hardware has not been tested.

Metal : MacBook with 96GB of RAM. Mac Studio class machines

NVIDIA CUDA : DGX Spark. RTX 6000 Pro

Install

git clone https://github.com/antirez/ds4
cd ds4

make

CLI

export CUDA_VISIBLE_DEVICES=0

./ds4 -m ./huihui-ai/Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF/Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-Q2.gguf \
  -p "Explain Redis streams in one paragraph."

Server

export CUDA_VISIBLE_DEVICES=0
./ds4-server \
  --cuda \
  -m ././huihui-ai/Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF/Huihui-DeepSeek-V4-Flash-BF16-abliterated-ds4-Q2.gguf \
  --ctx 131072 \
  --kv-disk-dir ./ds4-kv-cache \
  --kv-disk-space-mb 32768 \
  --power 75 \
  --warm-weights

curl test

curl http://127.0.0.1:8000/v1/models

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [
      {"role": "user", "content": "hello"}
    ],
    "temperature": 0.7,
    "max_tokens": 512,
    "stream": false
  }'

License

MIT. The base model copyright is held by DeepSeek; the GGUFs are redistributed under the base model's release terms.

Usage Warnings

  • Risk of Sensitive or Controversial Outputs: This model’s safety filtering has been significantly reduced, potentially generating sensitive, controversial, or inappropriate content. Users should exercise caution and rigorously review generated outputs.

  • Not Suitable for All Audiences: Due to limited content filtering, the model’s outputs may be inappropriate for public settings, underage users, or applications requiring high security.

  • Legal and Ethical Responsibilities: Users must ensure their usage complies with local laws and ethical standards. Generated content may carry legal or ethical risks, and users are solely responsible for any consequences.

  • Research and Experimental Use: It is recommended to use this model for research, testing, or controlled environments, avoiding direct use in production or public-facing commercial applications.

  • Monitoring and Review Recommendations: Users are strongly advised to monitor model outputs in real-time and conduct manual reviews when necessary to prevent the dissemination of inappropriate content.

  • No Default Safety Guarantees: Unlike standard models, this model has not undergone rigorous safety optimization. huihui.ai bears no responsibility for any consequences arising from its use.

Donation

If you like it, please click 'like' and follow us for more updates.
You can follow x.com/support_huihui to get the latest model information from huihui.ai.

Your donation helps us continue our further development and improvement, a cup of coffee can do it.

  • bitcoin（BTC):
  bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70ge
  • Support our work on Ko-fi (https://ko-fi.com/huihuiai)!
HUGGING FACE

上游数据

HF 下载量22.3万
HF 点赞量142

HF 下载量为源站最近 30 天统计，与本站点击分开。

SITE STATS

本站统计

下载点击—
请求数—
匿名去重—最近 7 天—

本站下载量记录按钮点击，不代表下载完成；短时连点会去重。

来源与版本

作者
huihui-ai
最近更新
2026/06/06
最新创建
2026/05/26
HF 趋势
556
数据快照
2026/09/16
版本
未知
许可证
mit