Fireworks AI just announced the release of Firefunction-v2, an open weights function calling model that takes function calling capabilities to the next level. Firefunction-v2 builds upon recent advancements in large language models to deliver a model optimized for real-world scenarios, including multi-turn conversation, instruction following, and parallel function calling.
| Feature | Firefunction-v1 | Firefunction-v2 | GPT-4o |
|---|---|---|---|
| Single turn function call (routing) | ✔️ | ✔️ | ✔️ |
| Multi-turn conversations | ⚠️ (limited) | ✔️ | ✔️ |
| Parallel function calling | ❌ | ✔️ | ✔️ |
| Instruction following | ⚠️ | ✔️ | ✔️ |
| General conversation (with optional function calling) | ⚠️ | ✔️ | ✔️ |
| Cost per 1M tokens | $0.5 | $0.9 | $5 (input), $15 (output) |
| Response latency | Up to 200 tokens/sec | ~180 tokens/sec | ~69 tokens/sec |
| Combined benchmark scores (MT bench, Gorilla, Nexus) | 0.49 | 0.81 | 0.80 |
It's been nearly a year since OpenAI introduced function calling as a feature, enabling language models to output structured text to call external APIs. While function calling has immense potential, productionizing this capability has been challenging due to the tradeoffs between open-source and closed-source models.
Traditional open-source function calling models tend to be overly specialized, focusing narrowly on function calling benchmarks at the expense of generalizability and general reasoning abilities. On the other hand, closed function calling models like GPT-4 and Claude offer strong performance in non-function calling tasks but come with high latencies and costs, limiting their production usability.
Fireworks AI took a novel approach to creating Firefunction-v2, based on user feedback highlighting the importance of real-world function calling models excelling at non-function calling tasks. Instead of overfitting to function calling scenarios, Fireworks augmented the Llama 3 70B base model with function calling capabilities while preserving its instruction following abilities.
The training process involved:
Fireworks evaluated Firefunction-v2 on a mix of publicly available datasets, including the Gorilla and Nexus benchmarks for function calling capability and MTBench for multi-turn instruction following. The results demonstrate that Firefunction-v2 achieves the highest performance on the medley of benchmarks, consistently outperforming Llama 3 on function calling tasks while maintaining similar multi-turn instruction following capabilities.
| Benchmark | Firefunction-v1 | Firefunction-v2 | Llama 3 70B Instruct | GPT-4o |
|---|---|---|---|---|
| Gorilla simple | 0.91 | 0.94 | 0.925 | 0.88 |
| Gorilla multiple_function | 0.92 | 0.91 | 0.86 | 0.91 |
| Gorilla parallel_function | 0 | 0.89 | 0.86 | 0.89 |
| Gorilla parallel_multiple_function | 0 | 0.79 | 0.62 | 0.72 |
| Nexus parallel | 0.38 | 0.51 | 0.30 | 0.47 |
| MTBench (multi-turn instruction following) | 0.73 | 0.84 | 0.89 | 0.93 |
| Average | 0.49 | 0.81 | 0.74 | 0.80 |
To showcase Firefunction-v2's real-world capabilities, Fireworks open-sourced a fully-functional chatbot demo that users can customize with their own functions. The demo app highlights some of the model's improved capabilities:
Firefunction-v2 demonstrates adeptness at more complex function calling tasks, reliably handling up to 30 function specs. In contrast, Firefunction-v1's performance degrades when utilizing more than ~5 functions. Parallel function calling, where the model executes two calls from one query, is crucial for real-world usage as it enables a more intuitive user experience and compatibility with a wider range of APIs.
Generalized models like Llama 3 often struggle to make intelligent decisions about when to call functions and may force unnecessary function calls. Firefunction-v2, on the other hand, correctly discerns when a function call is relevant to the given instruction, responding like a typical chat model when appropriate.
Ready to experience the power of Firefunction-v2? Get started with the model using Fireworks' comprehensive documentation, which includes sample apps and guides. Fireworks hosts Firefunction-v2 on their platform with a speed-optimized setup and an OpenAI-compatible API, making it easy to integrate the model into your existing projects.
You can also explore Firefunction-v2's capabilities in the UI playground, where you can add example functions and obtain code snippets for experimentation.
With the release of Firefunction-v2, Fireworks AI is committed to providing a model optimized for real-world usage in terms of response quality, speed, and cost. Developed with the feedback and support of the Fireworks community, Firefunction-v2 has already garnered enthusiastic reactions from beta testers impressed by its real-world readiness.
Fireworks AI invites developers to join their Discord function calling community and share feedback to shape the future of Firefunction models. As Fireworks continues to iterate on these models, they remain dedicated to helping developers productionize generative AI at scale, offering a platform that is fast, cost-efficient, and specialized to your use case.
With Firefunction-v2, the era of production-grade open-source AI is here. Happy building!
