How to Run Llama 3 70b with Ollama (and Easily Build AI Apps!)

In this comprehensive guide, we'll explore how to run the powerful Llama 3 70b language model locally using Ollama, and how to build custom AI apps without coding using Anakin AI. We'll cover the installation process, running Llama 3 with Ollama, creating AI apps with Anakin AI's no-code platform, and integrating AI capabilities into your projects using Anakin AI's APIs. By the end of this guide, you'll have the knowledge and skills to harness the capabilities of these cutting-edge AI tools.

Running Llama 3 70b with Ollama

Ollama is a free and open-source application that allows you to run various large language models, including Llama 3, on your own computer, even with limited resources. It takes advantage of the performance gains of llama.cpp, an open-source library designed to run LLMs locally with relatively low hardware requirements.

Installing Ollama

The first step is to install Ollama. It supports all major operating systems: Windows, macOS, and Linux. Follow the instructions on the official Ollama website (https://ollama.com/) to download and install the appropriate version for your system.

Windows Installation

1. Visit the Ollama website and download the Windows installer.
2. Run the installer and follow the on-screen instructions.
3. Once installed, you can launch Ollama from the Start menu or by running the ollama command in the terminal.

macOS Installation

1. Visit the Ollama website and download the macOS installer.
2. Double-click the downloaded file to extract the Ollama application.
3. Drag the Ollama application to your Applications folder.
4. Launch Ollama from the Applications folder or by running the ollama command in the terminal.

Linux Installation

1. Visit the Ollama website and download the Linux installer for your distribution.
2. Extract the downloaded archive.
3. Open a terminal and navigate to the extracted directory.
4. Run the install.sh script with sudo privileges: sudo ./install.sh
5. Once installed, you can run Ollama by typing ollama in the terminal.

Downloading and Running Llama 3 70b

Once Ollama is installed, open your terminal or command prompt and run the following command:

ollama run llama3:70b

This command will download and load the Llama 3 70b model, which is a large language model with 70 billion parameters. Depending on your internet connection and system specifications, this process may take some time.

After the download is complete, Ollama will launch a chat interface where you can interact with the Llama 3 70b model. You can ask questions, provide prompts, or engage in conversations, and the model will generate responses based on its training.

Optimizing Performance

If you experience slow performance or encounter issues running the 70b model due to limited system resources, you can try the following:

Use a smaller model: Ollama also provides access to the 8b version of Llama 3, which has fewer parameters and may run more efficiently on lower-end systems. To run the 8b model, use the command ollama run llama3:8b.

Use a smaller quantization: Ollama offers different quantization levels for the models, which can affect their size and performance. You can try running a smaller quantization level with the command ollama run llama3:70b-instruct-q2_K.

Upgrade your hardware: If you plan to work extensively with large language models like Llama 3 70b, consider upgrading your system's RAM and GPU to handle the computational demands more efficiently.

Building AI Apps with Anakin AI

Anakin AI is a no-code platform that allows you to create custom AI apps without writing a single line of code. It provides a user-friendly interface and integrates various AI models, including GPT-3, GPT-4, Claude, Stable Diffusion, and more.

Creating an Account

To get started with Anakin AI, visit their website (https://www.anakin.ai/) and create a free account. Once logged in, you'll have access to your personal workspace.

Building a Quick App

One of the simplest ways to create an AI app with Anakin AI is by using the "Quick App" feature. Here's how you can build a Quick App:

1. In your personal workspace, click on the "Add or Create an App" button.
2. Select the "Quick App" option and give your app a name.
3. Define your input fields, such as the topic for a blog post or a prompt for text generation.
4. Choose the AI model you want to use for your app.
5. Click "Generate" to test your app and see the output.

For example, let's create a Quick App that generates a blog post based on a given topic:

1. Name your app "Blog Post Generator."
2. Add an input field called "Topic" as a single-line text field.
3. Select the "GPT-4" model.
4. Click "Generate" and enter a topic, such as "Sustainable Living."
5. The app will generate a well-structured blog post on the given topic.

Customizing Your App

Anakin AI allows you to further customize your AI app by adding more input fields, adjusting the AI model parameters, and modifying the output format. Experiment with different settings to find the best configuration for your needs.

For example, you can add additional input fields like "Word Count" or "Tone" to the Blog Post Generator app, and adjust the AI model's parameters to generate content that matches your preferences.

Using Pre-made Apps

In addition to building your own apps, Anakin AI offers a library of pre-made AI apps created by developers and other users. You can browse and use these apps directly or clone them and customize them according to your requirements.

To use a pre-made app, follow these steps:

1. In your personal workspace, click on the "Explore Apps" button.
2. Browse through the available apps or use the search function to find specific apps.
3. Click on an app to view its details and description.
4. Click the "Use App" button to add the app to your workspace.
5. Customize the app's settings if desired, or use it as-is.

Sharing Your App

Once you've created a custom AI app that you're satisfied with, you can share it with others by publishing it to the Anakin AI App Store. This allows other users to discover and use your app, and you can earn credits when they sign up and utilize your app.

To share your app, follow these steps:

1. In your personal workspace, hover over the app you want to share and click the three dots.
2. Select "Manage" from the dropdown menu.
3. Click the "Publish to App Store" button.
4. Fill in the required information, such as the app's name, description, and icon.
5. Click "Publish" to make your app available in the App Store.

You can then share the link to your published app with others, and they can use it directly or clone it to their workspace for further customization.

API Integration with Anakin AI

Anakin AI offers comprehensive API services for all applications, empowering developers and organizations to seamlessly integrate and enhance their projects using Anakin AI APIs.

Advantages of API Integration

• Rapid Development: Develop AI applications tailored to your business needs using Anakin AI's intuitive visual interface, with real-time implementation across all clients.
• Flexibility: Support for multiple AI model providers, allowing you the flexibility to switch providers as needed.

• Streamlined Access: Pre-packaged access to the essential functionalities of the AI model.
• Stay Ahead: Stay ahead of the curve with upcoming advanced features available through the API.

How to Use Anakin AI's API to Build Apps

Step 1: Upgrade Your Plan and Check Your Account Credits

Anakin AI's API service is currently exclusively available to subscribers. While using the AI model through API calls, credits from your account balance will be consumed.

To check your subscription status or upgrade your plan, navigate to the Anakin AI Web App. Click on the avatar located in the lower left corner to access the Upgrade page. Please ensure that your current account has sufficient credits.

Step 2: Test Your App

Now, to test the app, select the app and click the "Generate" button. Confirm it runs properly and produces the expected output before proceeding.

Step 3: View API Documentation and Manage API Access Tokens

Next, visit the app Integration section at the top. In this section, you can click "View Details" to view the API documentation provided by Anakin AI, manage access tokens, and view the App ID.

Step 4: Generate Your API Access Token

Click the "Manage Token" button to manage your API Access token and select "New Token" to generate your API access token.

Complete the following token configuration, then click "Save and generate token," and finally, copy and save the API Access Token securely.

NOTE: The generated API access token will only be displayed once. Make sure to copy and save it securely right away. A best practice is to expose API keys in plaintext through backend calls rather than directly in frontend code or requests. This helps prevent abuse or attacks on your app.

You can create multiple API Access Tokens for an app to distribute it among different users or developers. This ensures that while users of the API can access the AI capabilities provided by the app developer, the underlying Prompt engineering and other tool capabilities remain intact.

How to Build a Quick App with Anakin AI

A Quick App allows you to generate high-quality text content such as blog posts, translations, and other creative content. By calling the Run a Quick App API, the user input content is sent, and the generated text result is obtained.

The model parameters and prompt template used to generate text depend on the input settings in the Anakin AI App -> Manage -> Design Page.

You can find the API documentation and request examples for the app in App -> Integration -> API List -> View Details.

Here's an API call example to create text completion information:

TIP: Please remember to replace the variable {{appId}} with the appId you want to request, and replace the ANAKINAI_API_ACCESS_TOKEN with the API Access Token you generated in Step 4.

curl --location --request POST 'https://api.anakin.ai/v1/quickapps/{{appId}}/runs' \
--header 'Authorization: Bearer ANAKINAI_API_ACCESS_TOKEN' \
--header 'X-Anakin-Api-Version: 2024-05-06' \
--header 'Content-Type: application/json' \
--data-raw '{
    "inputs": {
        "Product/Service": "Cloud Service",
        "Features": "Reliability and performance.",
        "Advantages": "Efficiency",
        "Framework": "Attention-Interest-Desire-Action"
    },
    "stream": true
}'

Check more details about the Quick App API in the API Reference.

How to Build an AI Chatbot App with Anakin AI

A Chatbot app lets you create chatbots that interact with users in a natural, question-and-answer format. To start a conversation, call the Conversation with Chatbot API, and continue to pass in the returned parameter name to maintain the conversation.

You can find the API documentation and sample requests for the app in App -> Integration -> API List -> View Details.

Here's an API call example to send conversation messages:

TIP: Please remember to replace the variable {{appId}} with the appId you want to request, and replace the ANAKINAI_API_ACCESS_TOKEN with the API Access Token you generated in Step 4.

curl --location --request POST 'https://api.anakin.ai/v1/chatbots/{{appId}}/messages' \
--header 'Authorization: Bearer ANAKINAI_API_ACCESS_TOKEN' \
--header 'X-Anakin-Api-Version: 2024-05-06' \
--header 'Content-Type: application/json' \
--data-raw '{
    "content": "What's your name? Are you the clever one?",
    "stream": true
}'

Conclusion

In this comprehensive guide, we've covered how to run the powerful Llama 3 70b language model locally using Ollama and how to build custom AI apps without coding using Anakin AI. By leveraging these tools, you can harness the capabilities of cutting-edge AI technologies and integrate them into your projects or workflows. Whether you're a developer, a business professional, or an AI enthusiast, Ollama and Anakin AI provide accessible and user-friendly solutions to unlock the potential of AI.