Web Design and Development
Architecture of a React app using a Node.js server to call an AI model

By the LaunchLen Web Solutions Team · Kitchener, Ontario · Last updated: November 9, 2026

Node.js AI integration usually starts simply: call a model API, show the answer. Then real users arrive. Responses are slow, requests fail, costs climb and someone pastes a secret into the front end.

This guide covers where AI calls should live in a Node.js app, how to stream answers to a React front end, when to use background jobs, and how to handle secrets, limits and failures.

Node.js AI integration: where the AI call should live

Keep AI calls on the server, not in the browser.

If the browser calls a model API directly, your API key is exposed to anyone who opens developer tools. You also lose control over what gets sent and how often.

A common shape looks like this:

  • The React front end sends the user’s request to your Node.js API.
  • Your server checks who the user is and what they’re allowed to do.
  • The server builds the prompt, adds approved context and calls the model API.
  • The server checks the result and sends back only what the user should see.

That server layer is also where tool use belongs. Our guide to LLM function calling explains why the model should only request actions, while your code decides whether to run them.

If you’re new to the runtime itself, our post on what Node.js is used for covers the basics.

Architecture of a React app using a Node.js server to call an AI model
Original graphic by LaunchLen Web Solutions Inc.

Streaming responses to a React front end

Model answers can take a while, so streaming sends the answer in pieces as it’s generated and users see progress instead of a spinner.

In Node.js, many model APIs can return a stream, and your server can pass those pieces on to the browser.

Two common ways to stream to the browser:

  • Server-sent events. A one-way stream from server to browser over standard HTTP. MDN documents the browser side.
  • Streaming fetch responses. The front end reads the response body as it arrives using the Streams API.

A few things to plan for:

  • Handle the user closing the tab, so you stop paying for an answer nobody will read.
  • Decide what happens if the stream breaks halfway.
  • Run final checks, such as filtering, on the complete answer when it matters.

Queues and background jobs for slow tasks

Not every AI task belongs in a live request. Summarizing a long document, processing a batch of invoices or running a multi-step agent can take too long.

For those, a background job pattern can help:

  • The API accepts the request and returns quickly with a job reference.
  • A worker process picks up the job from a queue and does the AI work.
  • The front end checks for the result, or the server notifies it when done.

Queues also help smooth out spikes: if many users submit work at once, jobs wait their turn instead of overwhelming the model API.

Caching can help too. If many users ask the same question about the same content, a cached answer can save time and cost. Be careful not to cache anything personal or user-specific across users.

Secrets, rate limits and retries

Secrets. Store API keys in environment variables or a secrets manager, never in code or the front end. OWASP’s secrets management guidance covers rotation, access control and auditing.

Rate limits. Model providers limit how many requests you can make. When you hit a limit, the API typically returns a “too many requests” status. MDN describes that status and the retry header that may come with it.

Retries. Retry temporary failures with a growing delay between attempts. Don’t retry forever, and don’t retry requests that failed because the input was bad.

Your own limits. Limit how often each user can call AI features. OWASP lists unbounded consumption as a risk for LLM applications, including runaway costs from unchecked use.

If your app runs on a VPS or similar server, choose hosting that fits the workload. Disclosure: LaunchLen has a referral relationship with Hostinger for hosting, VPS and self-hosted automation. Our recommendations stay based on your needs.

For new client builds we don’t need access to your systems to start. Credentials are needed only if we’re integrating an existing codebase, micro-SaaS or servers.

Testing and fallbacks

AI output varies, so tests need a slightly different approach.

Testing can include:

  • Unit tests for your prompt-building and response-checking code, with the model call mocked.
  • A set of real example questions run against the model before each release.
  • Checks that the app behaves well when the model API is slow, down or returns something unexpected.

Fallbacks keep the app usable when AI isn’t:

  • Show a clear message and let the user retry.
  • Fall back to a simpler feature, such as search results instead of a generated answer.
  • For customer-facing chatbots, hand off through the bottom-right website widget and let unresolved questions create a ticket for a specialist.

Log failures in a way that helps you debug without storing more personal data than you need.

FAQ

Can I call an AI model directly from React?

It’s not a good idea for production. Your API key would be exposed, and you lose control over usage and safety checks. Route calls through your Node.js server.

Should every AI feature stream its answer?

Not necessarily. Streaming suits chat-style answers. Short classifications or background jobs often don’t need it.

How do we keep AI costs under control?

Limit usage per user, cache safe repeated answers, keep prompts lean and move heavy work into queued jobs. Watch usage closely after launch.

Let’s work together on your project

Adding AI to an existing Node.js app, or planning a new one? We’re LaunchLen Web Solutions Inc.

Based in Kitchener, serving Waterloo Region and businesses across Ontario.

Our Software Development page explains custom software development in Kitchener-Waterloo and across Ontario: internal tools, customer-facing apps, integrations and AI features that connect to the systems you already rely on. These can be built with React on the front end and Node.js on the back end.

Before any build, we can agree the scope, deliverables and reporting with you. Book a free consultation to get started.

LLM Observability: Monitoring AI Features After Launch

Microsoft 365 or Google Workspace? How We Help Teams Decide

Leave a comment

Your email address will not be published. Required fields are marked *

DROP US A LINE

Connect with LaunchLen Web Solutions Inc.

Ready to take the first step towards unlocking opportunities, realizing goals, and embracing innovation? We're here and eager to connect.

To More Inquiry
+1 226 899 4813

Your Success Starts Here!