📊 View Lecture Slides Full-screen presentation with navigation

Session 11: Tool Calling & Actions

Session Duration: 2 Hours     Block: 3 — Agents, Evaluation & Deployment

Session clock

Minutes Mode Focus
0–50 Lecture Theoretical Foundation & Concepts
50–110 Core lab Calculator API & RAG Tooling
110–120 Checkpoint Pair share / show artifact

Note: Stretch work starts only after the Core checkpoint is completed.

Learning Objectives

By the end of this session, students will be able to:

  • Explain the mechanics of Function Calling (Tool Calling): how an LLM decides to request an action rather than outputting plain text.
  • Implement a bounded tool-execution loop (queryWithTools) in Node.js using a simple calculator warm-up.
  • Develop and Register a RAG semantic search function as a callable tool for the AI.
  • Architect dual endpoints in Express, keeping the standard POST /query intact while adding an experimental POST /tools endpoint.

Part 1: Theoretical Foundation — Function Calling

1.1 The Limitation of Pure Text

Until now, our LLMs have been confined to a box. They can read text (or context we inject via RAG) and output text. But what if the user asks: “What is 15% of $2,340?” or “What is the weather in London right now?”

An LLM is notoriously bad at math. And unless you scraped the weather forecast and injected it via RAG, it cannot know the current weather. To solve this, we give the model Hands—the ability to interact with external APIs and execute code. This is called Function Calling or Tool Calling.

1.2 How Function Calling Works (The Contract)

It is crucial to understand that the LLM does not execute code.

Here is the actual sequence of events:

  1. Declare: You send a prompt to the LLM, but you also include a JSON Schema describing a list of “Tools” you have available (e.g., calculator(a, b, operation)).
  2. Decide (LLM): The LLM reads the user prompt. It decides: “I need to do math to answer this. I will pause generating text and request the calculator tool.”
  3. Request: The API returns a special response object containing a functionCall request, complete with the arguments (e.g., a: 2340, b: 0.15, operation: multiply).
  4. Execute (Node.js): Your Node.js server intercepts this functionCall. Your server executes the actual JavaScript math function.
  5. Return: Your server sends the result (e.g., 351) back to the LLM API in a functionResponse.
  6. Synthesize (LLM): The LLM reads the result and finally generates the text: “15% of $2,340 is $351.”

1.3 Descriptions Dictate Behavior

Because the LLM relies entirely on your JSON Schema to understand the tools, your descriptions are as important as your code.

// BAD TOOL DECLARATION
description: 'Calculates numbers';

// GOOD TOOL DECLARATION
description: 'A secure calculator for basic arithmetic. Use this whenever the user asks a math question. Supports add, subtract, multiply, and divide.';

If the model doesn’t understand exactly when and how to use the tool based on the description, it will ignore it and try to guess the answer.

1.4 Security and the Bounded Loop

Never give an LLM an unrestricted eval() tool where it can write and execute arbitrary JavaScript or bash commands on your server. This is a massive security vulnerability. Furthermore, always bound your tool loop. An LLM might get stuck in an infinite loop, repeatedly calling the calculator incorrectly. Set a strict MAX_TOOL_CALLS = 4 to prevent endless API billing cycles.

Part 2: Practical Labs — Build the Tool Loop

Lab 11.1 — Calculator Warm-Up (Core)

Let’s wire up a safe tool loop. In your starter kit, open server.js. We have stubbed out an endpoint POST /tools which currently returns a 501 Not Implemented error.

  1. Import the provided tool logic: import { queryWithTools } from './lib/tools.js';
  2. Update the route logic:
app.post('/tools', async (req, res) => {
  try {
    const text = await queryWithTools(ai, req.body.query);
    return res.json({ response: text, sources: [] });
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});

Crucial architecture note: Do not replace Session 9’s RAG /query route. Keep them separate. We are adding a new capability, not destroying the old one.

  1. Open lib/tools.js and examine queryWithTools and safeCalculate.
  2. Test it via Postman or your UI (pointed at /tools): Ask _“What is (15 / 100) _ 2340?”* Note that you should never use a bare % sign in your test prompt to avoid parsing edge cases.
  3. Ask a non-math question: “Explain what an API is.” The model should bypass the tool and answer normally.

Lab 11.2 — RAG as a Search Tool (Core)

In Session 9, we forced the model to read RAG context on every single query. With tool calling, we can give the model a searchKnowledgeBaseTool and let it decide if it needs to search the vault.

  1. In lib/tools.js, locate the executeTool function branch for search_knowledge_base.
  2. Uncomment the wiring that calls retrieveRelevantChunks (you wrote this in Session 9). You will need to pass ctx.ai to it.
  3. In queryWithTools, add the searchKnowledgeBaseTool schema to the functionDeclarations array so the model knows it exists.
  4. Test it: Send a question related to your product vault. The model should invoke the search tool, your Node server should search the embeddings, return the text chunks to the model, and the model should synthesize the answer.
  5. Test the pivot: Immediately ask a pure math question. The model should recognize that the vault is useless for math and invoke the calculator instead.

Gate Check: If you cannot get the search tool working before the checkpoint, the calculator alone serves as the Core passing requirement for the warm-up, but you must finish the search tool before the final showcase.

Lab 11.3 — Checkpoint

  1. Open your terminal where the server is running.
  2. Capture the log output of a successful tool call (showing the functionCall request name, the arguments, and the result snippet returned by your code).
  3. Document this log block in your vault at 03-Project/prompts.md.

Stretch Goal: Add a third, domain-specific tool. (e.g., if your product is e-commerce, create a checkInventory tool that returns hardcoded JSON stock levels). Update your frontend UI to feature a toggle switch: “Ask Standard Agent” vs “Ask Tool-Enabled Agent”.

Key Takeaways

  • LLMs request tool execution via JSON; your Node.js application executes the code.
  • Tool schemas (descriptions and parameter definitions) are just as critical as prompt engineering.
  • Maintain separate architectural routes (e.g., /query vs /tools) to isolate experimental agentic features from stable RAG features.
  • Never grant an AI unrestricted execution rights (eval()).

Further Reading & Resources