> ## Documentation Index
> Fetch the complete documentation index at: https://together-ai-preview.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Llama 3.1

> Learn how to do function calling for Llama 3.1 models!

Llama 3.1 shipped natively with function calling support, but instead of specifying a `tool_choice` parameter like traditional function calling, it works with a special prompt syntax. Let's take a look at how to do function calling with Llama 3.1 models – strictly with a custom prompt!

> According to Meta, if you want to do a full conversation with tool calling, Llama 3.1 70B and Llama 3.1 405B are the two recommended options. Llama 3.1 8B is good for zero shot tool calling, but can't hold a full conversation at the same time.

## Function calling w/ Llama 3.1 70B

Say we have a function called `weatherTool` and want to pass it to LLama 3.1 to be able to call it when it sees fit. We'll define the function attributes, use the special prompt from [llama-agentic-system](https://github.com/meta-llama/llama-agentic-system) (from Meta) to pass the function to the model in the system prompt, and send in a prompt asking how the weather is in Tokyo.

<Tabs>
  <Tab title="python">
    ```
    import json
    from together import Together

    together = Together()

    weatherTool = {
    "name": "get_current_weather",
    "description": "Get the current weather in a given location",
    "parameters": {
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "The city and state, e.g. San Francisco, CA",
            },
        },
        "required": ["location"],
    },
    }

    toolPrompt = f"""
    You have access to the following functions:

    Use the function '{weatherTool["name"]}' to '{weatherTool["description"]}':
    {json.dumps(weatherTool)}

    If you choose to call a function ONLY reply in the following format with no prefix or suffix:

    <function=example_function_name>{{\"example_name\": \"example_value\"}}</function>

    Reminder:
    - Function calls MUST follow the specified format, start with <function= and end with </function>
    - Required parameters MUST be specified
    - Only call one function at a time
    - Put the entire function call reply on one line
    - If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls

    """

    messages = [
    {
        "role": "system",
        "content": toolPrompt,
    },
    {
        "role": "user",
        "content": "What is the weather in Tokyo?",
    },

    ]

    response = together.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
    messages=messages,
    max_tokens=1024,
    temperature=0,
    )

    messages.append(response.choices[0].message)
    print(response.choices[0].message.content)
    ```
  </Tab>

  <Tab title="TypeScript">
    ```
    import Together from 'together-ai';

    const together = new Together();

    const weatherTool = {
    name: 'get_current_weather',
    description: 'Get the current weather in a given location',
    parameters: {
    type: 'object',
    properties: {
      location: {
        type: 'string',
        description: 'The city and state, e.g. San Francisco, CA',
      },
    },
    required: ['location'],
    },
    };

    const toolPrompt = `You have access to the following functions:

    Use the function '${weatherTool.name}' to '${weatherTool.description}':
    ${JSON.stringify(weatherTool)}

    If you choose to call a function ONLY reply in the following format with no prefix or suffix:

    <function=example_function_name>{"example_name": "example_value"}</function>

    Reminder:
    - Function calls MUST follow the specified format, start with <function= and end with </function>
    - Required parameters MUST be specified
    - Only call one function at a time
    - Put the entire function call reply on one line
    - If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls

    `;

    let messages = [
    {
      role: 'system',
      content: toolPrompt,
    },
    {
      role: 'user',
      content: 'What is the weather in Casablanca?',
    },
    ];

    async function main() {
    const response = await together.chat.completions.create({
    model: 'meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo',
    messages: messages,
    max_tokens: 1024,
    temperature: 0,
    });
    if (response.choices?.[0]?.message) {
    messages.push({
      role: response.choices[0].message.role,
      content: response.choices[0].message.content!,
    });
    console.log(response.choices?.[0]?.message?.content);
    }
    }

    main();
    ```
  </Tab>
</Tabs>