Newsletter
Stay Ahead
Occasional, engineering-led notes on applied AI — what we're building, running, and learning in production.
Occasional, engineering-led notes on applied AI — what we're building, running, and learning in production.
Use case: Build a RAG chatbot that cites your own documentation
Retrieval-augmented generation lets a chat model answer questions from your documentation instead of guessing from whatever it happened to see during training. Webanto's inference API gives you both halves of the pipeline: POST /api/v1/embeddings (model nomic-embed-text) turns your docs into vectors for retrieval, and POST /api/v1/chat (model qwen, a self-hosted 35B multimodal model) turns the retrieved passages into a grounded answer with a citation back to the source page. Webanto does not store or index the vectors for you — you keep them in pgvector, Pinecone, or whatever store you already run, and re-embed the user's question at query time to find the nearest chunks. Answer quality tracks your chunking and retrieval choices far more than the chat model itself, so budget real time for both instead of assuming the model alone determines quality.
Answers grounded in your actual documentation, with the source page cited alongside each response, rather than the model inventing plausible-sounding but wrong information.
Consistent answers to the same underlying question regardless of how it's phrased, because retrieval matches on meaning rather than exact keywords.
A support team that can point users at a doc update instead of manually rewriting the same answer every time a question repeats.
A visible audit trail — which chunks were retrieved for a given answer — that makes wrong answers debuggable instead of treating the whole system as a black box.
Split your documentation into chunks — roughly 300-800 tokens each with some overlap, splitting on headings or sections so every chunk stays self-contained.
Call POST /api/v1/embeddings with model "nomic-embed-text" for every chunk. 20,000 chunks ÷ 100 per credit = 200 credits, comfortably inside a Growth plan's 50,000 monthly credits.
Store each returned vector alongside its chunk text and source URL in your own vector store (pgvector, Pinecone, etc.) — Webanto returns vectors only, it does not host an index for you.
At query time, embed the visitor's question with the same nomic-embed-text call and run a nearest-neighbour search against your store to pull the top 5-8 matching chunks.
Assemble a prompt with the retrieved chunks plus the question and send it to POST /api/v1/chat with model "qwen", instructing it to answer only from the supplied context and cite the source URL for each claim.
Chat billing runs on tokens: a typical RAG turn (context plus question plus answer) lands around 800-1,500 tokens, so roughly 1.5-3 turns per credit at 2,000 tokens per credit.
Test with questions you already know the answer to, check the cited sources actually match the claim, and tighten the 'answer only from context' instruction if the model starts drifting into unsupported claims.
Yes — RAG reduces hallucination but does not eliminate it. If retrieval pulls an irrelevant chunk, or the question has no good match in your docs, Qwen can still generate a fluent but wrong answer. Explicitly instruct it to say 'not found in the documentation' when the retrieved context doesn't cover the question, and test that instruction under adversarial phrasing before trusting it in production.
No. POST /api/v1/embeddings returns a vector for whatever text you send, and Webanto does not retain a queryable copy of it. You're responsible for storing the vectors and running similarity search yourself, in pgvector, Pinecone, Weaviate, or any store you prefer.
More than most people expect. Chunks that are too large dilute the embedding with unrelated content and retrieval gets fuzzy; chunks that are too small lose context and the chat model can't answer fully from them. Splitting on natural document boundaries and keeping overlap between adjacent chunks usually beats fixed-length splitting.
Yes, Qwen is multimodal and accepts images directly in the chat call, so you can pass a relevant diagram alongside the retrieved text chunks when it's likely to help the answer. That doesn't replace embedding the text itself, though — image content still needs to be described or embedded separately before retrieval can find it.
The same rate as the first pass — 100 chunks per credit — but you only need to re-embed the chunks that actually changed, not the whole set, if you track a content hash per chunk and skip the ones that are unchanged.
Shoppers who spot a product in the wild, on someone else's feed, or on a competitor's site rarely know the right keywords to find it in your catalogue. Visual search closes that gap: a shopper uploads
Large media libraries accumulate redundancy fast — the same product shot re-exported at a different crop, a stock photo uploaded twice under different filenames, a banner re-saved at a lower compressi
Reverse image search answers a narrower question than catalogue browsing: given one photo someone just handed you — a screenshot, a competitor's product shot, a customer's snapshot of a damaged item —
Start a 14-day free trial on any paid plan. No credit card required.