Recipe · Voice-driven UI

Register an ElevenLabs Client Tool

Let an ElevenLabs voice agent invoke functions that run inside the visitor's browser — open cards, scroll to sections, play videos — by wiring client-side tool calls to the page's JavaScript.

Time: ~25 min Difficulty: intermediate Companion skill: register-elevenlabs-client-tool

Why client tools

A voice agent that only talks is a chatbot with audio. A voice agent that can change the page it's running on is a UI layer. Client tools are the bridge: the agent decides to do something, the platform passes the invocation to the browser, and the page executes it.

This is different from server-side tools. Server tools hit an API, process data, and return a result. Client tools manipulate the DOM, play media, or trigger navigation. Both are useful; this recipe is about the client side.

What you'll build

Prerequisites

Step 1 — Define the client tool

Create a new tool via the ElevenLabs API. The critical field is "type": "client" — this tells the platform the function runs in the browser, not on a server.

curl -X POST https://api.elevenlabs.io/v1/convai/tools \
  -H "xi-api-key: $ELEVENLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "client",
    "name": "scroll_to_section",
    "description": "Scroll the page to a named section. Call this when the user asks to navigate to pricing, docs, or a specific feature.",
    "parameters": {
      "type": "object",
      "properties": {
        "section": {
          "type": "string",
          "description": "The section ID to scroll to. Must be 'pricing', 'docs', 'features', or 'contact'."
        }
      },
      "required": ["section"]
    }
  }'

Save the returned tool ID — you'll attach it to the agent in Step 2.

Description quality matters. The agent decides whether to call this tool based on the description. Write it like you're explaining the function to a junior developer, not to a machine. "Call this when the user asks to navigate to..." is better than "Scrolls page."

Step 2 — Attach to the agent and update the prompt

Patch the agent to include the tool and add trigger instructions to the system prompt:

curl -X PATCH https://api.elevenlabs.io/v1/convai/agents/{agent_id} \
  -H "xi-api-key: $ELEVENLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "conversation_config": {
      "agent": {
        "prompt": {
          "system_prompt": "You are a helpful assistant embedded in a website. When the user asks to see pricing, documentation, features, or contact information, use the scroll_to_section client tool. Do not just describe the section — scroll to it."
        }
      },
      "client_tools": ["tool-id-from-step-1"]
    }
  }'

Step 3 — Handle the call in the frontend

The frontend listens for client_tool_call events from the SDK and executes the matching function. The exact event shape depends on which SDK you're using.

Using the direct ElevenLabs SDK

import { Conversation } from '@ elevenlabs/elevenlabs-js';

const conversation = new Conversation({
  agentId: 'your-agent-id',
  onMessage: (message) => {
    if (message.type === 'client_tool_call') {
      handleClientTool(message);
    }
  }
});

function handleClientTool(message) {
  const { tool_name, parameters } = message;

  switch (tool_name) {
    case 'scroll_to_section':
      document.getElementById(parameters.section)?.scrollIntoView({ behavior: 'smooth' });
      break;
    case 'open_modal':
      document.getElementById(parameters.modal_id)?.showModal();
      break;
    default:
      console.warn('Unknown client tool:', tool_name);
  }
}

Using the LiveAvatar SDK

import { LiveAvatarSession } from '@heygen/liveavatar-web-sdk';

const session = new LiveAvatarSession({
  token: '...',
  videoElement: document.getElementById('avatar'),
  onClientToolCall: (data) => {
    // LiveAvatar nests the tool call differently
    const toolCall = data.client_tool_call || data;
    handleClientTool(toolCall);
  }
});
Gotcha — tool-call nesting. The LiveAvatar SDK nests the tool call under data.client_tool_call. The direct ElevenLabs SDK puts it at the top level of the message. Your handler must check both shapes or it will silently miss calls.

Step 4 — Set expects_response correctly

For LiveAvatar integrations, set "expects_response": "false" on the tool definition. Client tools through LiveAvatar cannot return values to the agent (the pipeline is one-way), so telling the agent to wait for a response will cause a timeout.

curl -X PATCH https://api.elevenlabs.io/v1/convai/tools/{tool_id} \
  -H "xi-api-key: $ELEVENLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "expects_response": false
  }'

Server-side effects (database writes, API calls) should not happen in client tools. If you need a side effect, use a server-side tool instead, or have the client tool call an internal API endpoint.

Step 5 — Add a fallback watcher

The primary event channel can drop tool calls under load or on mobile networks. A fallback scan of transcription events catches missed invocations:

// Fallback: if the agent says it will scroll but the event never fired,
// parse the transcript and trigger manually.
conversation.onMessage((message) => {
  if (message.type === 'transcript' && message.role === 'agent') {
    const text = message.text.toLowerCase();
    if (text.includes('scroll') && text.includes('pricing')) {
      // Check if scroll_to_section already fired
      if (!document.getElementById('pricing').classList.contains('scrolled-to')) {
        document.getElementById('pricing').scrollIntoView({ behavior: 'smooth' });
      }
    }
  }
});

This is defensive, not primary. The real fix is reliable event delivery, but the fallback prevents the agent from claiming it did something the user never saw.

Step 6 — Verify end-to-end

  1. Open the agent in your page. Start a conversation.
  2. Say: "Show me the pricing."
  3. The agent should respond with something like "Scrolling to pricing now..." and the page should smoothly scroll to the pricing section.
  4. Check the browser console. You should see the client_tool_call event logged with tool_name: "scroll_to_section" and the correct section parameter.
  5. Test a miss — say something unrelated. The tool should not fire.

What this gets you

When not to use this

Server-side effects belong in server tools, not client tools. If the agent needs to write to a database, send an email, or charge a card, use a server tool with a webhook. Client tools should only manipulate the DOM or trigger client-side behavior.

What's next

Companion materials

Seth Shoultes builds at garagedoorscience.com and writes about it at sethshoultes.com/blog.