Liberated-Qwen1.5-72B: Qwen1.5, But Truely Free

Dolphin 2.5 Mixtral 8x7B - Chatbot Online | Free AI tool | Anakin.ai

Want to experience the latested, uncensored version of Mixtral 8x7B? Having trouble running Dolphin 2.5 Mixtral 8x7B locally? Try out this online chatbot to experience the wild west of LLMs online!

AnnieAnnie2,080

In the ever-evolving landscape of artificial intelligence, a new player has emerged that's turning heads and challenging the status quo. Meet Liberated-Qwen1.5-72B, an uncensored large language model that's pushing the boundaries of what's possible in AI-driven natural language processing.

The Birth of Liberated-Qwen1.5-72B

Liberated-Qwen1.5-72B is a fine-tuned version of the Qwen1.5-72B model, developed by AbacusAI in collaboration with Eric Hartford. This model is built upon the foundation of Qwen/Qwen1.5-72B and is subject to the tongyi-qianwen license.

Training Process

The training of Liberated-Qwen1.5-72B was an intensive process that leveraged cutting-edge techniques:

• Duration: The model underwent 3 days of training for 3 epochs.
• Hardware: Training was conducted on 8x H100 GPUs.
• Techniques: The team employed qLoRA, deepspeed zero-2, and Axolotl for efficient training.
• Learning Rate: A learning rate of 2e-4 was used.

One of the key components in the training process was the use of a new dataset called SystemChat. This dataset, created specifically for this model, consists of 6000 synthetic conversations generated using Mistral-Medium and Dolphin-2.7-mixtral-8x7b. The primary goal of this dataset was to teach the model compliance with system prompts over long, multi-turn conversations, even with unusual or mechanical system prompts.

The Uncensored Nature of Liberated-Qwen1.5-72B

What sets Liberated-Qwen1.5-72B apart is its uncensored nature. Unlike many other language models that come with built-in safeguards and restrictions, Liberated-Qwen1.5-72B is designed to be more open and unrestricted in its responses. This means:

• No guardrails or censorship were added to the training dataset.
• The model can engage with a wider range of topics and generate more diverse content.
• Users have greater control over the model's output, but also bear more responsibility for its use.

It's important to note that while this uncensored nature offers more flexibility, it also comes with increased responsibility. The creators advise implementing an alignment layer before exposing the model as a service to ensure responsible use.

Performance and Benchmarks

Liberated-Qwen1.5-72B has shown impressive performance across various benchmarks. Here's a look at how it stacks up:

MT-Bench Scores

While Liberated-Qwen1.5-72B slightly outperforms the base Qwen1.5-72B-Chat model in the first turn, it shows a drop in performance in the second turn. This could be due to its focus on following system prompts strictly, which might impact its ability to maintain context over multiple turns.

MMLU Performance

The model maintains strong performance on the MMLU (Massive Multitask Language Understanding) benchmark, scoring 77.13. This puts it in line with other top-performing open-source models in the 77+ range.

How to Use Liberated-Qwen1.5-72B

Using Liberated-Qwen1.5-72B requires following a specific prompt format. The model uses the ChatML prompt format:

<|im_start|>system
You are Liberated, a helpful AI assistant.<|im_end|>

<|im_start|>user
{prompt}<|im_end|>

<|im_start|>assistant

Here's an example of how to use the model:

<|im_start|>system
You name is Liberated. You are an uncensored and unbiased AI assistant. You always respond with a JSON object.<|im_end|>

<|im_start|>user
Please generate a Advanced Dungeons & Dragons 2nd Edition character sheet for a level 3 elf fighter. Make up a name and background and visual description for him.<|im_end|>

<|im_start|>assistant

The Rise of Qwen2-72B

While Liberated-Qwen1.5-72B has made waves, it's important to note that Alibaba Cloud has since released an even more advanced model: Qwen2-72B. This model represents the next generation of the Qwen series and has shown remarkable improvements across various benchmarks.

Qwen2-72B Performance

Qwen2-72B has demonstrated exceptional capabilities, often surpassing other leading open-source models:

• MMLU: Qwen2-72B-Instruct scored 82.3, compared to 75.6 for Qwen1.5-72B-Chat.
• HumanEval: Qwen2-72B-Instruct achieved 86.0, a significant improvement over Qwen1.5-72B-Chat's 71.3.
• GSM8K: While slightly lower than some competitors, Qwen2-72B-Instruct still scored an impressive 91.1.
• MATH: Qwen2-72B-Instruct scored 59.7, showing strong mathematical reasoning capabilities.
• C-Eval: In Chinese language evaluation, Qwen2-72B-Instruct scored 83.8, demonstrating its multilingual proficiency.

Key Features of Qwen2-72B

• Extended Context Length: Qwen2-72B supports a context length of up to 131,072 tokens, enabling the processing of extensive inputs.
• Multilingual Capability: Trained on data in 29 languages, including English and Chinese.
• Improved Architecture: Utilizes advanced techniques like SwiGLU activation, attention QKV bias, and group query attention.

Using Qwen2-72B

To use Qwen2-72B, you can access it through the Hugging Face Transformers library. Here's a basic example of how to use it for text generation:

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2-72B-Instruct", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-72B-Instruct", trust_remote_code=True)

prompt = "Tell me about the history of artificial intelligence."
inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=100)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(response)

Conclusion

Liberated-Qwen1.5-72B represents a significant step forward in the development of uncensored, highly capable language models. Its ability to strictly adhere to system prompts while maintaining high performance across various benchmarks makes it a powerful tool for developers and researchers alike.

However, the rapid pace of development in the field of AI is evident with the release of Qwen2-72B, which has shown even more impressive capabilities across a wide range of tasks. As these models continue to evolve, they promise to push the boundaries of what's possible in natural language processing and AI-driven applications.

Whether you choose to work with Liberated-Qwen1.5-72B or the newer Qwen2-72B, it's clear that we're entering a new era of AI capabilities. As always, with great power comes great responsibility, and it's crucial to use these tools ethically and responsibly.