Implementing AI agents from Scratch using Langchain and OpenAI

AI agents within LangChain take a language model and tie it together with a set of tools to address larger, more complex tasks. Unlike a static chain of instructions, an agent dynamically decides at each step which action (tool) to take based on the conversation and intermediate results.

In practice, this means the agent prompts the LLM (the "reasoning engine") to formulate its next action or question, potentially invoke a tool (like a web search or calculator), and then use whatever results show up as new information in its reasoning. The agent continues to loop until it develops a final answer.

A LangChain agent is an LLM-based system that can decide actions dynamically, such as calling tools or answering directly. Unlike a chain (fixed steps), an agent reasons step-by-step, adapting based on context.

If your task is always the same process that follows a fixed sequence, a chain would suffice (a hard-coded sequence). If you would like to have flexibility which would allow operate with different tools or follow a more branching logic, you would need an agent. For instance, agents are useful for a process where you want the agent to be able to search the web, interact with a knowledge base, do something computational, and then summarize these steps in a single seamless process.

This guide will cover the basics and follow the building of a LangChain agent in detail step-by-step. This guide will cover the primary components (tools, LLMs, prompts), how the agent loop works, and best practices to create more robust agents. The examples will use the most current LangChain API (2025 version), and it is expected that the reader is familiar with Python and large language models (LLMs).

• Follows a loop: Think → Act → Observe → Repeat.
• The LLM decides whether to act or respond.
• Tool results are fed back as context for the next step.

Like a detective solving a case:

• Uses a notebook (scratchpad) for thoughts.
• Chooses from a toolbox (APIs/functions).
• Stops when confident in the answer.

• Tools:
• External functions or APIs with names and descriptions.
• Used by the agent to perform tasks (e.g., search, math).
• LLM:
• The decision-making model (e.g., GPT-4o-mini, Gemini 2.0 ).
• Chooses the next action or gives the final answer based on tthe ool outputs.
• Prompt/Scratchpad:
• Guides LLM for proper tool usage, its guardrails and clear tool differentiation.
• Stores previous actions and results for context.

A tool is simply a Python function wrapped with metadata. For example, to make a calculator tool that evaluates arithmetic expressions, you might write:

from langchain. tools import Tool

def calculate_expression(expr: str) -> str:

try:

result = eval(expr)

return str(result)

except Exception as e:

return f"Error: {e}"

def return_dummy_weather(city: str) -> str:

return f"The weather in {city} is cloudy"

calc_tool = Tool(

name="Calculator",

description="Performs simple arithmetic. Input should be a valid Python expression, e.g. '2+2'.",

func=calculate_expression

)

# Dummy weather tool

weather_tool = Tool(

name="WeatherSearch",

description="Tells current weather of a city. Input should be a valid city in string, e.g 'paris'.",

func=calculate_expression

)

This calc_tool tells the agent that whenever it needs to do math, it can call the tool "Calculator" with an input string. The agent's prompt will include this tool's name and description (and optionally how to format the input). The description should be clear and specific – vagueness can confuse the agent, causing it to choose the wrong tool or misuse it.

LangChain comes with many built-in tools and tool wrappers. For example:

• Web Search Tool: Interfaces like TavilySearchResults or GoogleSerperAPIWrapper let an agent search the web. (You'll need API keys for external search services.)
• Retriever Tool: Wraps a vector database or document store. As described in one example, you might first load documents and create a retriever, then use create_retriever_tool to expose it to the agent. This tool fetches relevant text snippets from your data given the query.
• Custom API Tools: You can define your tool that calls any API. For instance, a weather_tool calling a weather API, or a jira_tool that creates JIRA tickets. The agent only needs the Python function; LangChain handles calling it when the agent decides.

When giving tools to an agent, we put them in a list:

tools = [calc_tool, weather_tool, search_tool, retriever_tool, ...]

The agent will see this list (typically as part of the prompt, or via the Tool objects) and may choose among them.

** Each tool should ideally perform a clear, atomic function. Complex or multi-step logic can confuse the agent. If needed, break tasks into simpler tools or chains and let the agent sequence them. **

The LLM (often a chat model) processes prompts and generates next steps. In LangChain 2025, a common import is:

from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(model_name="gpt-4o", temperature=0.0)

You may also use ChatAnthropic, ChatGooglePalm, etc. Setting temperature=0 (or low) can make the agent's decisions more deterministic, which is often desirable for tool use. You'll pass this LLM to the agent.

The agent prompt template defines how the LLM is instructed to behave. A common pattern (ReAct-style) is to include:

• System/Instruction : Explains to the assistant that it is an agent with certain tools. Example: "You are an agent designed to answer questions. You have access to the following tools…"
• Tool Descriptions : Lists each tool's name and description, so the model knows what actions it can take.
• Format Guide : Tells the model how to output its reasoning. For example, it might use a structured JSON or markdown format like you can also use libraries like Pydantic to get more precise and formatted JSON objects for tool calls.

Example Prompt based on our Calculator tool.

<Persona>

  You are a helpful, precise AI assistant capable of solving user queries using available tools.

  You can perform reasoning, fetch information, and carry out calculations when needed.

</Persona>

<Guardrails>

  - Only call a tool if it's required to answer the question.

  - Do not guess values or fabricate information.

  - Never perform code execution or arithmetic by yourself; use the Calculator tool for all such tasks.

</Guardrails>

<AvailableTools>

  <Tool>

    <Name>Calculator</Name>

    <Description>

      Performs simple arithmetic. Input must be a valid Python expression, such as '3 * (4 + 5)'.

      Use this tool only for basic math operations (e.g., +, -, *, /, parentheses).

    </Description>

    <Format>

      To use this tool, return:

      Action: Calculator

      Action Input: 2 + 2

    </Format>

  </Tool>

  <Tool>

    <Name>Weather</Name>

    <Description>

      Tells current weather of a city. Input should be a valid city in string, e.g 'paris'.

    </Description>

    <Format>

      To use this tool, return:

      Action: Weather

      Action Input: Paris

    </Format>

  </Tool>

</AvailableTools>

Under the hood, an agent uses a loop to repeatedly query the LLM, parse its output, execute tools, and update context. Conceptually:

1. Initial Input : The user's question is given to the agent (and any system instructions).
   
   
2. LLM Response : The agent prompts the LLM, which returns either a final answer or an action.
   
   
3. Tool Invocation (if any) : If the output is an action, the agent executes the corresponding tool function with the provided input. (E.g. ***calc_tool(query="22") )
   
   
4. Observe : The result from the tool (text, JSON, etc.) is captured. The agent adds this result to the scratchpad.
   
   
5. Loop or End : The agent checks if the LLM signaled a final answer or if any stopping criteria (max steps/time) are met. If not finished, it goes back to step 2: it calls the LLM again, now including the new observations in the prompt. This continues, building up a chain of reasoning.
   
   
6. Return Answer : Once the agent decides it's done, it returns the final answer to the user.
   
   

This process is illustrated by the pseudocode in the LangChain source (simplified):

from langchain.schema import HumanMessage, AIMessage, SystemMessage

def process_with_tool_loop(user_input: str):

    MAX_ITERATIONS = 10

    current_iteration = 0

    messages = [

        SystemMessage(content="You are a helpful assistant with access to a calculator tool."),

        HumanMessage(content=user_input)

    ]

    while current_iteration < MAX_ITERATIONS:

        print(f"Iteration {current_iteration + 1}")

        response = llm.invoke(messages)

        # Check if LLM wants to call a function

        if not response.additional_kwargs.get("function_call"):

            print(f"Final answer: {response.content}")

            break

        function_call = response.additional_kwargs["function_call"]

        function_name = function_call["name"]

        function_args = function_call["arguments"]

        # Execute the tool

        if function_name == "Calculator":

            import json

            args = json.loads(function_args)

            tool_result = calculate_expression(args.get("expr", ""))

        if function_name == "WeatherSearch":

            import json

            args = json.loads(function_args)

            tool_result = weather_tool(args.get("city", ""))

        # Add function call and result to conversation.

        messages.append(response)

        messages.append(AIMessage(content=f"Function result: {tool_result}"))

        current_iteration += 1

    return response.content

When building AI chat systems, preserving conversation history is essential for providing contextual, coherent responses. The ConversationHistoryService handles this by transforming stored messages into LangChain-compatible message objects that the model can understand. This formatting is especially important when using OpenAI models, as LangChain standardizes the message structure (e.g., HumanMessage, AIMessage, ToolMessage) required for proper tool invocation and response handling.

Different models may expect varying formats for tool calls and conversation history. For other LLMs, such as Gemini, the history format may differ especially when supporting Agentic behavior, so the message transformation logic must be adapted to match each model’s specific input requirements.

This system:

• Handles multiple sender types (USER, AI, TOOL)
• Ensures messages are properly ordered and valid according to OpenAI LLM ( gpt-4o-mini ) requirements.
• Constructs an array of Langchain messages starting with the system prompt

We must store the complete conversation history along with the tool call and tool response in the Database, then at every LLM call, we should fetch the history from the DB and formulate that according to the langchain / LLM requirement.

For example :

from langchain_core.messages import HumanMessage, AIMessage, ToolMessage

def convert_to_langchain_message(message, next_message=None):

    sender_type = message.get("sender_type")

    if sender_type == "TOOL":

        return ToolMessage(

            tool_call_id=message.get("tool_call_id"),

            name=message.get("content"),

            content=message.get("content")

        )

    elif sender_type == "USER":

        return HumanMessage(content=message.get("content"))

    else:  # Assume AI

        if next_message is None:

            return None

        if message.get("additional_metadata", {}).get("tool_calls") and next_message.get("sender_type") != "TOOL":

            return None

        return AIMessage(

            content=message.get("content"),

            additional_kwargs=message.get("additional_metadata", {})

        )

• It loops through stored conversation messages and based on the sender_type, it converts each into the appropriate LangChain message:
• TOOL ➜ ToolMessage
• USER ➜ HumanMessage

Otherwise (typically AI) ➜ AIMessage

Building robust agents often requires attention to detail and thoughtful configuration. Here are some tips and advanced considerations:

• Clear Tool Description ****: The agent relies heavily on the text descriptions of tools. Make these descriptions concise and unambiguous. For example, specify input/output formats or any usage constraints. Poor descriptions can cause the agent to pick the wrong tool or misuse a tool.
• Zero-Shot vs Few-shot Prompts ****You can provide examples in the system prompt to guide the agent's reasoning (few-shot prompting), especially if the default behavior is incorrect. For example, give one or two example interactions showing how to use each tool.
• Control Temperature ****: Use a low temperature (e.g. 0.1 or 0.2) for agents to make consistent decisions. High randomness can lead to inconsistent tool use.

Set Iteration Limits : To avoid infinite loops, configure the agent executor's limits. LangChain's AgentExecutor has parameters max_iterations (default 10) and max_execution_time to halt runaway loops.

LangChain makes it surprisingly straightforward to build intelligent agents by combining LLM reasoning with tool usage. By defining clear tools and prompt instructions, you can create a system that handles multi-step questions and leverages external data or computation.

Remember that agents are powerful but also require careful crafting of prompts, descriptions, and limits to behave reliably. Whether you're building a QA chatbot that searches the web, an analytics assistant that processes databases, or any autonomous tool-based LLM system, understanding the agent loop and its components is key.

With the foundations in this guide, you can start designing your LangChain agents and explore more advanced topics like multi-agent coordination or integration with LangGraph for complex pipelines. Happy agent-building!

Stay tuned for an exciting deep dive into building AI Agents using Google Gemini!

In the next blog, we will explore how to leverage Gemini's powerful multimodal capabilities to create intelligent, tool-using agents that can reason, act, and adapt to complex tasks — all in real time.