Why API Pagination Cursors Repeat or Skip Results
Debug cursor pagination by checking stable sort order, duplicate timestamps, cursor encoding, filters, page size, inserts and client retries.
Quick Answer
Cursor pagination repeats or skips results when the cursor does not encode a stable position, the sort order is not deterministic, filters change between requests, new records are inserted during paging, or the client retries with an old cursor. Use a stable sort key and treat the cursor as an opaque server value.
Example Scenario
A list endpoint returns 50 items per page. Users report seeing the same item twice or missing one item when they click next. The response includes a next_cursor, but results are sorted only by updated_at, and several records share the same timestamp. New records also arrive while the user is paging through the list.
Step-by-Step Explanation
- Capture the first page, cursor value and second page together.
- Check whether the sort order is stable and includes a tie-breaker.
- Keep filters and page size identical across cursor requests.
- Treat cursor values as opaque and do not edit or decode them in the client.
- Check how inserts, deletes and updates behave during paging.
- Log cursor metadata server-side without exposing private query details.
Cursor Pagination Needs a Stable Position
A cursor is a marker for where the next page should continue. That marker only works when the server can define a stable order. Sorting by a non-unique field such as updated_at, score or name can create ambiguity when multiple rows share the same value.
A stable cursor often includes a tie-breaker such as id along with the primary sort value. For example, updated_at plus id is more stable than updated_at alone. Without the tie-breaker, the server may not know which records with the same timestamp were already sent.
When debugging repeats or skips, capture the boundary items around the page transition. The last item of page one and the first item of page two usually reveal whether the cursor resumed at the right position.
Filters Must Stay the Same
A cursor is usually valid for a specific query: filters, sort, page size and sometimes user context. If the client changes a filter while reusing an old cursor, the server may continue from a position that no longer makes sense.
This happens when UI state changes between requests but the pagination state is not reset. A user changes status from open to closed, yet the client keeps next_cursor from the open query. The next page may look empty, duplicated or unrelated.
Reset cursors whenever query parameters that affect the result set change. Include those parameters in logs when investigating pagination bugs.
Opaque Means Do Not Edit It
Some cursors are Base64-encoded JSON. Others are encrypted or signed. The client should treat them as opaque values unless the API documentation says otherwise. Decoding a cursor for curiosity is fine in a safe environment, but editing it can break integrity or create undefined behavior.
Do not build a cursor by copying a timestamp from the last item unless the API contract explicitly uses that model. Server-generated cursors may include internal tie-breakers, filters or signatures that the client cannot reconstruct correctly.
If a cursor looks URL-unsafe, encode it as a query parameter value. Do not strip padding, plus signs or slashes unless the API documents that format.
Changing Data Makes Paging Harder
Records can be inserted, deleted or updated while a user is paging. That can move items earlier or later in the sort order. Cursor pagination handles this better than offset pagination, but only when the cursor and sort are designed for the mutation pattern.
For activity feeds, decide whether pagination should represent a snapshot or a moving view. A snapshot avoids surprises but requires more state. A moving view may show newly inserted items but needs careful duplicate handling.
If exact once-only traversal matters, use server-side snapshot tokens or export jobs rather than a live cursor over changing data.
Client Retries Can Reuse Old Cursors
A retry after timeout can accidentally append the same page twice. The server may have returned the page, but the client did not receive the response. If the client retries and then appends both responses, duplicates appear even though the API behaved correctly.
Track page request ids or cursor values on the client. Before appending results, check whether that cursor was already applied. For id-based lists, de-duplicate by stable item id as a final guard.
Do not hide server pagination bugs with client de-duplication alone. Use de-duplication for resilience, but still investigate repeated boundary items.
Production Checks
Log the cursor presence, page size, sort keys, filter hash and first or last item id for each page request. Avoid logging full cursor contents if they include private query data.
Add tests with duplicate sort values, changed filters, inserted rows between pages and retry of the same cursor. These cases catch most real pagination regressions.
Use JSON Compare to compare adjacent page responses and confirm whether the repeated or missing record is at the boundary, inside the page, or caused by client appending behavior.
Expose a safe debug mode in non-production that returns the decoded cursor metadata. That helps backend and frontend teams confirm sort keys and filter hashes without teaching clients to depend on cursor internals.
For customer-facing exports, avoid live cursor traversal when exact completeness matters. Use an export snapshot or job id so inserts and updates during traversal do not change the result set.
Pagination bugs are easier to diagnose when each item includes a stable id in logs and test output. If only display names are compared, duplicates can be missed because two records share the same label.
Check backwards pagination separately from forward pagination. previous_cursor is not always the inverse of next_cursor, especially when sort direction changes or when cursors encode boundary operators.
For APIs consumed by mobile clients, consider offline and resume behavior. A cursor saved yesterday may be invalid today if filters, permissions or data retention changed. Return a clear cursor expired error rather than a misleading empty page.
If the API supports page size changes, decide whether a cursor created with page size 50 can be reused with page size 100. Many systems should reject that combination because the cursor was created for a specific traversal shape. Silent reuse can create overlaps that are hard to reproduce.
For analytics or billing screens, prefer deterministic exports over live pagination when users need totals to reconcile exactly. Interactive browsing and audit-grade extraction have different consistency requirements.
Code Examples
function applyFilter(nextFilter) {
state.filter = nextFilter;
state.cursor = null;
state.items = [];
loadNextPage();
}const url = new URL('/api/orders', location.origin);
url.searchParams.set('cursor', nextCursor);
url.searchParams.set('status', statusFilter);
await fetch(url);const seen = new Set(existingItems.map(item => item.id));
const merged = [...existingItems];
for (const item of page.items) {
if (!seen.has(item.id)) merged.push(item);
}Common Mistakes
- Sorting by a non-unique field without a tie-breaker.
- Reusing a cursor after filters or page size changed.
- Editing decoded cursor contents in the client.
- Ignoring inserts and updates that happen during paging.
- Appending retry responses without checking duplicate page state.
FAQ
Should clients decode cursors?
Usually no. Treat cursors as opaque unless the API explicitly documents their structure.
Why do duplicate timestamps break pagination?
A timestamp alone may not identify a unique position, so records with the same timestamp can repeat or be skipped.
Should I reset pagination when filters change?
Yes. A cursor usually belongs to one specific query shape.
Can new records appear while paging?
Yes. APIs need a defined policy for live data, snapshots or duplicate handling.