You built the perfect automation. The logic is sound, the mappings are clean, and the test run returned a beautiful JSON payload. But a week later, you notice a discrepancy. Your CRM has 5,000 new leads, but your automation only processed 50.<\/p>\n\n\n\n
You just hit the Pagination Wall<\/b>.<\/p>\n\n\n\n
Pagination is the method APIs use to break large datasets into manageable chunks, or “pages.” This preserves server performance. For a human browsing a website, clicking “Next Page” is easy. For an automated workflow in Make.com automation<\/a><\/b>, it determines if your system works or fails.<\/p>\n\n\n\n If you are building self-driving ecosystems rather than simple tasks\u2014the core mission of Thinkpeak.ai<\/a>\u2014incomplete data is unacceptable. An AI agent that only sees the first 50 rows of your sales data isn’t intelligent. It is hallucinating based on a partial reality.<\/p>\n\n\n\n In this guide, we move beyond basic documentation. We will explore how to architect robust pagination workflows<\/b> in Make.com. We cover everything from simple logic to complex cursor-based navigation used by tools like Shopify and Slack.<\/p>\n\n\n\n Before dragging modules onto your canvas, you must identify how your specific API handles data. Open your API documentation<\/a> and look for “Pagination.” You will typically find one of three models.<\/p>\n\n\n\n This is the most intuitive method. You tell the API to give you “page 3” or to skip the first 200 records and give the next 100.<\/p>\n\n\n\n High-volume platforms like Meta, Stripe, and Slack use cursor-based pagination<\/b> to prevent data drift. The API provides a unique “token” pointing to the last item you received. You must pass this token back to get the next batch.<\/p>\n\n\n\n The API doesn’t put pagination details in the JSON body. Instead, it hides the “Next Page URL” inside the HTTP Response Headers<\/b>.<\/p>\n\n\n\n Many users ignore the advanced settings in Make\u2019s standard HTTP > Make a request<\/strong> module. They are unaware that native pagination support exists.<\/p>\n\n\n\n When to use it:<\/strong> Use this for simple, standard APIs. This works best when you just need to increment a query parameter until the data stops.<\/p>\n\n\n\n How to configure:<\/strong><\/p>\n\n\n\n This reveals three new fields:<\/p>\n\n\n\n The “Condition” Formula:<\/strong><\/p>\n\n\n\n You usually check if the response body is empty.<\/p>\n\n\n\n As long as this formula is TRUE, Make will keep requesting the next page.<\/p>\n\n\n\n Why this often fails:<\/strong> The native feature is opaque. It is hard to debug why a loop stopped. For enterprise-grade reliability, we recommend building the logic explicitly using the methods below.<\/p>\n\n\n\n This is the gold standard for controlling your data flow. It separates the counting logic from the fetching logic. This works best for offset pagination<\/b>.<\/p>\n\n\n\n Step 1: The “Scout” Request<\/strong><\/p>\n\n\n\n First, make a single call to the API to get the total count of records.<\/p>\n\n\n\n Step 2: Calculate the Loops<\/strong><\/p>\n\n\n\n Kullan\u0131m Set Variable<\/strong> module to calculate how many pages you need.<\/p>\n\n\n\n Step 3: The Repeater<\/strong><\/p>\n\n\n\n Add the Flow Control Repeater<\/b> Mod\u00fcl.<\/p>\n\n\n\n Step 4: The Paginated Request<\/strong><\/p>\n\n\n\n Add your main HTTP mod\u00fcl\u00fc<\/strong> after the Repeater.<\/p>\n\n\n\n Step 5: Aggregation<\/strong><\/p>\n\n\n\n Do not process the data inside the loop immediately. Add an Array Aggregator<\/b> after the HTTP module.<\/p>\n\n\n\n Sonu\u00e7:<\/strong> The Repeater runs 50 times. The Aggregator waits for all 50 runs to finish. Then, it releases a single bundle containing all 5,000 leads. You can now map this massive array to your database in one go.<\/p>\n\n\n\n Thinkpeak.ai Pro Tip:<\/strong> If you are dealing with massive datasets (10k+ rows), the Array Aggregator might hit Make\u2019s memory limit. In that case, our Bespoke Engineering team uses streaming architectures to process data in batches.<\/p>\n\n\n\n Cursor pagination is tricky because you do not know how many pages exist. You must “keep going” until the API stops giving you a Mimari:<\/strong> We use a Repeater set to a high safety limit. We pass the “Cursor” token from one iteration to the next using Make\u2019s variable storage.<\/p>\n\n\n\n Step 1: Initialize the Token<\/strong><\/p>\n\n\n\n Step 2: The Safety Loop<\/strong><\/p>\n\n\n\n Step 3: Retrieve the Current Token<\/strong><\/p>\n\n\n\n Inside the loop, grab the token from the previous run.<\/p>\n\n\n\n Step 4: The Make.com API <\/a>Call<\/strong><\/p>\n\n\n\n Step 5: Update the Token<\/strong><\/p>\n\n\n\n Step 6: The “Break” Logic<\/strong><\/p>\n\n\n\n If the API returns no token, you must stop the Repeater.<\/p>\n\n\n\n This method works, but it is complex to debug. If you prefer a plug-and-play solution, check the templates at Thinkpeak.ai<\/a>.<\/p>\n\n\n\n When you paginate efficiently, you create a new problem: speed. You might hit the API 50 times in 2 seconds. This triggers a 429 Too Many Requests<\/b> error.<\/p>\n\n\n\n The Solution: The Sleep Directive<\/strong><\/p>\n\n\n\n This tells Make to wait 60 seconds and try again automatically if the API blocks you.<\/p>\n\n\n\nThe Three Types of API Pagination<\/h2>\n\n\n\n
1. Offset or Page-Based (The Classical Model)<\/h3>\n\n\n\n
\n
page=1<\/code>, limit=100<\/code> OR offset=200<\/code>, limit=100<\/code>.<\/li>\n\n\n\n2. Cursor-Based (The Modern Standard)<\/h3>\n\n\n\n
\n
starting_after=xh52...<\/code>, next_page_token=abc...<\/code>.<\/li>\n\n\n\n3. Link Header (The “Hidden” Model)<\/h3>\n\n\n\n
\n
Method 1: The Native HTTP Module Solution<\/h2>\n\n\n\n
\n
\n
Sayfa<\/code>).<\/li>\n\n\n\nEvet<\/code>. It merges your pagination params with your original request.<\/li>\n\n\n\n{{length(body.data) > 0}}<\/code><\/p>\n\n\n\nMethod 2: The Repeater Loop<\/h2>\n\n\n\n
Scenario: Fetching 5,000 leads, 100 at a time.<\/h3>\n\n\n\n
\n
...\/leads?limit=1<\/code><\/li>\n\n\n\n\"total\": 5000<\/code>.<\/li>\n<\/ul>\n\n\n\n\n
{{ceil(1.data.total \/ 100)}}<\/code><\/li>\n\n\n\n\n
\n
...\/leads<\/code><\/li>\n\n\n\nlimit<\/code> i\u00e7in 100<\/code> ve Sayfa<\/code> to the i<\/code> value from the Repeater module.<\/li>\n<\/ul>\n\n\n\n\n
data[]<\/code> array from your HTTP module.<\/li>\n<\/ul>\n\n\n\nMethod 3: Handling Cursor Pagination<\/h2>\n\n\n\n
next_token<\/code>. Make.com lacks a native Do-While loop<\/b>, so we must engineer one.<\/p>\n\n\n\n\n
next_cursor<\/code><\/li>\n\n\n\n\n
\n
next_cursor<\/code><\/li>\n<\/ul>\n\n\n\n\n
cursor<\/code> to the value from the Get Variable module.<\/li>\n<\/ul>\n\n\n\n\n
next_cursor<\/code><\/li>\n\n\n\nnext_page_token<\/code> from the HTTP response body.<\/li>\n<\/ul>\n\n\n\n\n
next_cursor<\/code> (from the response) Does Not Exist.<\/li>\n\n\n\nHandling Rate Limits<\/h2>\n\n\n\n
\n
Blueprints for Success: Thinkpeak.ai\u2019s Approach<\/h2>\n\n\n\n