Function Calling

largelanguagemodel

Function calling is the ability of an LLM to effectively and efficiently call external tools and APIs and interact with them.

LLMs like GPT4 and GPT3.5 from OpenAI have been fine tuned to detect when to call different functions and then output JSON containing arguments to call the function. These act as tools Automatic Reasoning and Tool Use (ART) in the AI application and we can use more than one tool in one request. Like calling external API from natural language to complete a certain task.

Function Calling with GPT-4

when a user asks what weather is like in Hyderabad, the LLM alone cannot answer this question. This can be answered by integrating an external tool with the LLM.

to handle such an request we need to pass the functions which can be called by the LLM in the OpenAI API request like

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "unit": {
                        "type": "string", 
                        "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },   
    }
]

the getget_current_weather function returns the temperature of the location. When we pass this function definition this function is not actually called but rather returns the JSON required to call the function

here is a completion function

def get_completion(messages, model="gpt-3.5-turbo-1106", temperature=0, max_tokens=300, tools=None):
    response = openai.chat.completions.create(
        model=model,
        messages=messages,
        temperature=temperature,
        max_tokens=max_tokens,
        tools=tools
    )
    return response.choices[0].message

here is the user question

messages = [
    {
        "role": "user",
        "content": "What is the weather like in Hyderabad?"
    }
]

we can call the get_completions function in this manner

response = get_completion(messages, tools=tools)

the response variable

ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='...', function=Function(arguments='{"location":"London","unit":"celsius"}', name='get_current_weather'), type='function')])

the arguments in has all the information extracted by the LLM to be passed to the function and complete the request

we can then choose to call an external API for actual weather. once the response is ready we can pass it back to the model to answer the user question.

References

• promptingguide.ai