Correct MaxItems docs about where NextToken is returned

[hxperl]

What is wrong

The generated PaginationConfig docs describe MaxItems like this:

MaxItems (integer) – The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.

The pages yielded by the iterator never carry that token. PageIterator records a resume token on itself (_truncate_response → self.resume_token, and the page-boundary branch in __iter__), and only build_full_result() copies it out under a NextToken key. A page dict contains whatever pagination token the service returned, and the service only returns one when it was asked for a partial page — which PaginationConfig controls via PageSize (the paginator's limit_key), not via MaxItems. MaxItems is applied client-side after the response arrives and is never sent to the service.

So a reader who follows the sentence above and indexes the token on a page gets a KeyError. That is what boto/boto3#3677 reports.

Reproduction

The issue's repro needs S3 credentials, so here is the same thing against a stub, using botocore's own Paginator on this branch's parent commit. The fake service holds 10 items and returns a NextToken only when it has more to give:

from botocore.paginate import Paginator

class FakeShape:
    class _M: type_name = "integer"
    members = {"MaxResults": _M()}
class FakeModel:
    input_shape = FakeShape()

ALL = list(range(1, 11))

def method(**kwargs):
    limit = kwargs.get("MaxResults")
    start = int(kwargs["NextToken"]) if kwargs.get("NextToken") else 0
    end = len(ALL) if limit is None else min(start + limit, len(ALL))
    page = {"Items": ALL[start:end]}
    if end < len(ALL):
        page["NextToken"] = str(end)
    return page

CONFIG = {"output_token": "NextToken", "input_token": "NextToken",
          "result_key": "Items", "limit_key": "MaxResults"}

def show(label, **pagconf):
    print(f"--- {label}: PaginationConfig={pagconf}")
    it = Paginator(method, CONFIG, FakeModel()).paginate(PaginationConfig=pagconf)
    for n, pg in enumerate(it):
        print(f"    page[{n}] Items={pg['Items']}  'NextToken' in page: {'NextToken' in pg}")
    print(f"    -> iterator.resume_token: {it.resume_token!r}")
    full = Paginator(method, CONFIG, FakeModel()).paginate(
        PaginationConfig=pagconf).build_full_result()
    print(f"    -> build_full_result(): keys={sorted(full)}\n")

show("MaxItems=4, no PageSize", MaxItems=4)
show("PageSize=4, no MaxItems", PageSize=4)
show("MaxItems=4 AND PageSize=4", MaxItems=4, PageSize=4)

Output:

--- MaxItems=4, no PageSize: PaginationConfig={'MaxItems': 4}
    page[0] Items=[1, 2, 3, 4]  'NextToken' in page: False
    -> iterator.resume_token: 'eyJOZXh0VG9rZW4iOiBudWxsLCAiYm90b190cnVuY2F0ZV9hbW91bnQiOiA0fQ=='
    -> build_full_result(): keys=['Items', 'NextToken']

--- PageSize=4, no MaxItems: PaginationConfig={'PageSize': 4}
    page[0] Items=[1, 2, 3, 4]  'NextToken' in page: True
    page[1] Items=[5, 6, 7, 8]  'NextToken' in page: True
    page[2] Items=[9, 10]  'NextToken' in page: False
    -> iterator.resume_token: None
    -> build_full_result(): keys=['Items']

--- MaxItems=4 AND PageSize=4: PaginationConfig={'MaxItems': 4, 'PageSize': 4}
    page[0] Items=[1, 2, 3, 4]  'NextToken' in page: True
    -> iterator.resume_token: 'eyJOZXh0VG9rZW4iOiAiNCJ9'
    -> build_full_result(): keys=['Items', 'NextToken']

The first block is the documented condition — 10 items available, MaxItems=4 — and there is no NextToken on the page. The second block shows the inverse: PageSize alone puts tokens on pages but leaves build_full_result() without one, because pagination ran to completion.

The change

One DocumentedShape documentation string in botocore/docs/paginator.py. Rendered result:

MaxItems (integer) – The total number of items to return. If the total number of items available is more than the value specified, a NextToken is provided in the dictionary returned by build_full_result(), and on the resume_token attribute of the page iterator, that you can use to resume pagination. Individual pages yielded by the iterator carry only the pagination tokens the service itself returned, which are affected by PageSize rather than by MaxItems.

Judgement calls, and where I would welcome a different answer

• Scope. I changed only the MaxItems string, because that is the sentence the issue quotes and links by line. I deliberately left two adjacent things alone: the PageSize string ("The size of each page."), which says nothing about it being the thing that drives server-side paging and token emission, and the NextToken entry in botocore_pagination_response_params, which renders under Response Syntax and so still reads as if every page carried the token. Both arguably belong in the same cleanup. I kept them out to keep this reviewable; say the word and I will fold them in, or open them separately. The PageSize constraint gap that @RyanFitzSimmonsAK raised is RDS describe_db_instances paginator has incorrect PaginationConfig description boto3#3798 and is untouched here.
• Wording. Naming build_full_result() and resume_token in per-service generated docs is more API detail than the surrounding text usually carries. The alternative is a vaguer "the token is not returned on individual pages" without saying where it is, which I thought less useful. Happy to trim if you prefer the shorter form.
• I did not touch the paginators user guide, which the issue also suggests improving.

What I verified, and what I did not

• python -m pytest tests/unit/docs tests/unit/test_paginate.py -q → 323 passed on this branch, and 323 passed on the same commit without the change. The existing assertion in tests/unit/docs/test_paginator.py matches the - **MaxItems** *(integer) --* header line only, not the body text, so it is unaffected.
• I rendered the doc through tests/unit/docs' own harness to confirm the reST output above.
• Not run: tests/functional/docs. It renders documentation for every service and I had to abandon it on a disk-constrained machine, so CI should be the judge there. Everything above is macOS arm64, Python 3.14, against e2b579b.

Refs boto/boto3#3677.

---

Generated by AI tools, and reviewed by hxperl.