Connect n8n to Local LLMs (Ollama)

Connecting n8n to Local LLMs (Ollama): The 2026 Sovereign AI Stack

For years, the AI narrative focused on a rental model. Businesses rent intelligence from OpenAI, Anthropic, or Google. They pay per token and often trade data privacy for convenience. As we settle into 2026, the focus is shifting back toward Sovereign AI.

Powerful open-weights models like Llama 3 and efficient runners like Ollama have changed the game. Local AI is now a viable enterprise strategy. It is no longer just for hobbyists. It is for businesses that want to build self-driving ecosystems without leaking intellectual property.

At Thinkpeak.ai, we help businesses build these autonomous systems. Whether you use our templates or commission bespoke tools, running logic locally is powerful. It reduces costs, ensures 100% data privacy, and speeds up operations.

This guide acts as a technical blueprint. We will show you how to connect n8n workflow automation to Ollama. We will cover infrastructure, networking challenges, and advanced workflows.

—

The Business Case for Local Intelligence

Why should a business manage its own inference infrastructure in 2026? The decision usually comes down to three factors: Privacy, Cost, and Control.

1. Data Sovereignty & GDPR Compliance

Many enterprises cite security as a primary barrier to AI adoption. When you send customer data or code to a public API, you expose that data to a third party.

By running Ollama locally, your data never leaves your private server. For industries like Finance and Healthcare, this level of data privacy is a requirement, not a luxury.

2. The “Zero-Marginal Cost” Employee

Cloud APIs charge for every single token. If you run a high-volume agent, those bills add up quickly. A local LLM costs the same whether it processes one lead or one million.

The only cost is electricity. Once you buy the hardware, your digital employee works for free. This creates a model of zero-marginal cost for labor.

3. Latency and “Edge” Automation

Network round-trips to external servers take time. Local inference happens at the speed of your hardware.

For internal tools, like a bulk uploader that cleanses thousands of rows, local models offer speed. Edge automation provides a snappy, real-time experience that cloud APIs often struggle to match.

—

Prerequisites: The Sovereign Stack

You need a specific environment to follow this guide. Unlike cloud software, local AI relies heavily on your machine’s specifications.

Hardware Requirements (2026 Standards)

• Minimum: 16GB RAM, NVIDIA GPU with 8GB VRAM (e.g., RTX 3060/4060). This is capable of running Llama 3 8B.
• Recommended: 32GB+ RAM, NVIDIA GPU with 24GB VRAM (e.g., RTX 3090/4090). This runs larger models comfortably.
• Apple Silicon: M2/M3/M4 Max chips with 32GB+ Unified Memory are excellent for inference.

Software Stack

• Docker: The standard for running n8n.
• Ollama: The local LLM runner. It abstracts complexity and provides a simple API.
• n8n: We recommend the self-hosted Docker version for maximum control.

—

Step 1: Installing and Configuring Ollama

Ollama is the standard for local inference. It mimics the OpenAI API structure. This makes it easy to swap cloud models for local ones.

1. Install Ollama:

# For Linux/WSL2
curl -fsSL https://ollama.com/install.sh | sh

For Windows or Mac, simply download the installer from the official website.

2. Pull Your Model:

For automation, we need models that follow instructions well. Llama 3 is the gold standard for general automation. Mistral is excellent for speed.

ollama pull llama3
ollama pull mistral

3. Verify the API:

Ollama listens on port 11434 by default. Verify it is running by visiting your localhost port in a browser. You should see a message stating “Ollama is running”.

—

Step 2: The Networking “Gotcha” (Crucial for Docker Users)

This is where most users fail. If you run n8n inside a Docker container, it cannot see `localhost` on your host machine by default. Inside the container, `localhost` refers to the container itself.

The Solution: `host.docker.internal`

You must configure Docker networking correctly. This allows n8n to talk to Ollama on your host machine.

If using docker run:

docker run -it --rm 
 --name n8n 
 -p 5678:5678 
 --add-host=host.docker.internal:host-gateway 
 -v n8n_data:/home/node/.n8n 
 docker.n8n.io/n8nio/n8n

If using `docker-compose.yml` (Recommended):

version: '3.8'

services:
  n8n:
    image: docker.n8n.io/n8nio/n8n
    ports:
      - "5678:5678"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - n8n_data:/home/node/.n8n

Adding extra_hosts maps host.docker.internal to your machine’s IP address. Now, n8n can reach Ollama.

—

Step 3: Connecting n8n to Ollama

We recommend two methods for integration. Use Method A for standard text generation. Use Method B for advanced control over parameters.

Method A: The Native “Ollama Chat Model” Node

n8n has excellent native AI support. This is the plug-and-play method.

1. Open your n8n workflow canvas.
2. Add the “Basic LLM Chain” node.
3. Connect the Ollama Chat Model node to the Model input.
4. Credentials: Create a new credential. Use http://host.docker.internal:11434 as the Base URL.
5. Model Name: Type llama3.
6. Execute the node to generate text locally.

Method B: The HTTP Request Node (Advanced Control)

Sometimes you need more control. You can interact directly with the Ollama API using an HTTP Request node.

1. Add an HTTP Request node.
2. Method: POST
3. URL: http://host.docker.internal:11434/api/chat
4. Body Parameters (JSON):

{
  "model": "llama3",
  "messages": [
    {
      "role": "user",
      "content": "Analyze this email: {{ $json.email_body }}"
    }
  ],
  "format": "json",
  "stream": false,
  "options": {
    "temperature": 0.1,
    "seed": 42
  }
}

Note the "format": "json" parameter. This forces the model to output valid JSON, which is crucial for automation.

—

Advanced Workflow: The “Privacy-First” RAG Agent

One powerful application is Retrieval-Augmented Generation (RAG). This allows you to chat with your own documents without uploading them to the cloud.

1. Vector Store (The Brain)

You need a local database for document embeddings. We recommend Qdrant or Postgres. These can run in Docker containers alongside n8n.

2. The Ingestion Workflow

• Read Binary Files: Ingest your PDFs.
• Text Splitter: Chunk text into manageable pieces.
• Ollama Embeddings: Connect this to the Embeddings input. Use a model like nomic-embed-text.
• Vector Store Node: Insert the documents into your database.

3. The Retrieval Workflow

• AI Agent Node: This acts as the orchestrator.
• Vector Store Tool: Connect your database as a tool.
• Ollama Chat Model: Power the agent with Llama 3.

The result is a system where zero data leaves your server. You can query contracts or internal wikis securely.

—

Structuring Unstructured Data

The value of AI is structuring chaos. Transforming messy emails into clean database rows is essential. Local LLMs used to struggle with strict JSON, but JSON Mode has solved this.

The “Inbound Lead Qualifier” Implementation

Imagine receiving a free-text form submission about a budget and team size.

The n8n Prompt:

You are a data extraction engine. Extract the following fields from the user input.
Output ONLY valid JSON.

Input: {{ $json.message }}

Required Fields:
- company_size (integer)
- budget (integer)
- intent (string: "high", "medium", "low")

By enforcing this structure, you can route the output to a Switch Node. This transforms static data into dynamic action.

—

Model Selection Guide: Which Brain to Use?

Not all local models are equal. You must balance speed, intelligence, and VRAM usage. Here is our model selection guide.

—

The Hybrid Architecture: When to use Local vs. Cloud

We often advocate for Hybrid Architectures. It is rarely an all-or-nothing decision.

When to use Local (Ollama):

• High Volume: Categorizing thousands of support tickets.
• Sensitive Data: Processing PII or financial statements.
• Offline Environments: Systems with air-gapped security.

When to use Cloud (GPT-4o):

• One-shot Creativity: Writing high-stakes sales proposals.
• Complex Reasoning: Solving edge cases that stump smaller models.
• Visual Analysis: OCR reliability for complex documents.

—

Troubleshooting Common Issues

1. “Connection Refused”

Ensure you are using http://host.docker.internal:11434. Check that you added the host gateway to your Docker config. Verify that firewalls are not blocking the port.

2. Slow Inference / Timeouts

If n8n times out, increase the Timeout setting in the HTTP Request node. Local models can be slow on CPUs. Ensure GPU offloading is enabled.

3. “Context Window Exceeded”

Llama 3 has a context window limit. If you pass massive PDFs, the model will fail. Use a text-splitter node in n8n to chunk data before processing.

—

Conclusion: Build Your Own Ecosystem

Connecting n8n to local LLMs via Ollama is a strategic move. It allows you to build Custom AI Agents that work privately and cost-effectively.

You now have the foundation for a self-driving business. You can process data and automate outreach without paying rent on your innovation. The real magic lies in the workflows you build on top of this infrastructure.

Ready to transform your operations?

We are your partner in this transition. From instant templates to full-stack custom development, we help you build the proprietary software stack of the future.

Browse Automation Templates    Book a Discovery Call →

—

Frequently Asked Questions (FAQ)

Can I use Ollama with n8n Cloud?

Technically, no. n8n Cloud cannot access your local computer’s localhost. You would need to expose your local instance via a tunnel like ngrok. For security, we recommend self-hosting n8n.

Does Llama 3 support Function Calling in n8n?

Yes. Ollama supports tool calling. You can define tools like a calendar or calculator in the AI Agent node. The model will recognize when to use these tools.

How do I handle GPU memory with multiple workflows?

Ollama handles model loading dynamically. It will swap models in and out of VRAM. This causes slight latency. We recommend sticking to one versatile model for high-traffic environments.

Resources

• n8n – Ollama Integration Guide: https://docs.ollama.com/integrations/n8n
• Ollama Credentials Documentation: https://docs.n8n.io/integrations/builtin/credentials/ollama/
• Ollama Official Website: https://ollama.com
• n8n Official Website: https://n8n.io
• Mistral AI Official Website: https://mistral.ai