Skip to content

MCP

Building an MCP server for internal tools: what we got wrong first

Our first server mirrored our REST API and the model used it badly. Five changes that made the difference.

6 min readByteWeave Studio

Our first MCP server was a thin wrapper over an existing REST API. Every endpoint became a tool, the descriptions came from the OpenAPI summaries, and it took an afternoon. It worked in the sense that calls succeeded, and it was close to useless in practice — the model chained four calls to answer questions a person would have asked in one.

The rewrite was mostly deletion. What follows is what changed.

Design tools around tasks, not endpoints

A REST API is decomposed for programmers who will read documentation and compose calls deliberately. A tool list is read once, quickly, by something deciding what to do next. Those are different audiences and they want different granularity.

Our API had list-customers, get-customer, list-orders and get-order. The model needed "find this customer’s recent orders", which was four calls and a join it sometimes got wrong. Replacing them with one task-shaped tool removed the failure entirely. The rule we settled on: if answering a common question needs more than two calls, that is a missing tool.

The description is the prompt

Tool descriptions inherited from an API summary are written for someone who already knows the domain. "Returns order objects filtered by status" tells a model nothing about when to reach for it.

Write descriptions that say when to use the tool, what it returns, and when to prefer a different one. Name the units and the formats — that a date is ISO, that an amount is in paise, that the identifier is the internal one and not the invoice number. Every ambiguity you leave gets resolved by guessing.

Return less than you think

Our first version returned full JSON objects because that is what the API returned. A twelve-order response filled a large part of the context window with keys nobody needed, and quality dropped for reasons that had nothing to do with the tool being wrong.

Return the fields that answer the question, formatted compactly, with a pointer for fetching detail if it turns out to be needed. Paginate, and say in the response that more exists rather than silently truncating — a model that does not know it saw a partial list will happily state a total.

Make errors instructions

A tool returning "400 Bad Request" gives the model nothing to act on, and it will usually retry the same call. An error that says which argument was invalid, what form was expected, and what a valid example looks like gets corrected on the next attempt.

We treat error text as part of the interface now, and write it the way you would write a message to a colleague who cannot see your code. It is the cheapest reliability improvement we found.

Separate reading from writing

Our first server exposed read and write tools under a single credential, which meant every session carried the authority to modify production data whether or not the task needed it.

Now the read tools and the write tools are separate, with separate scopes, and anything destructive requires explicit confirmation rather than being callable directly. This has cost us nothing in capability. It is worth deciding before you need it, because the first time a model calls a delete tool on a misread instruction is a bad moment to be designing the policy.

  • MCP
  • Model Context Protocol
  • Tooling
  • API design
  • LLM

More reading

Have a problem
worth solving?

Tell us what you're building. We'll help you figure out what's possible — and say so if we're not the right people for it.

Start a conversationSee the work