PyPI — Python Package Index

Overview

Covers the full read workflow on PyPI: searching for packages, reading package detail pages (description, version history, download files), and retrieving metadata via the JSON API. The JSON API is the recommended approach for automated workflows — no CAPTCHA, no browser required.

Not covered here: Package upload/publishing, account management, user profiles, API tokens.

Auth required for: Uploading packages, managing releases. All browsing and search are public.

Agent Paths

Programmatic [API] — No browser needed

• Package metadata + all versions: GET https://pypi.org/pypi/{name}/json
• Specific version metadata: GET https://pypi.org/pypi/{name}/{version}/json
• Search: No clean JSON search API — use browser or navigate directly to /project/{name}/

Required headers: None — all public endpoints are open

Browser [Browser] — Requires browser control

• Use selectors and Navigation Flows below
• Required when: searching without knowing the package name, browsing download files, reading classifiers/maintainer info

When to use which

• Default to Programmatic if you know the package name — one call returns everything
• Use Browser when: you need to search for a package by description (not name), or the task involves downloading/copying the pip install command
• Avoid browser search (/search/) — hCaptcha blocks automated browsers. Use the JSON API instead.

Agent Knowledge Note

Most frontier LLMs have PyPI's API and package structure in training data. This skill adds value via:

• hCaptcha on /search/ — automated browsers are consistently blocked on the search results page. Training data may suggest using the search UI; the correct path is the JSON API or direct /project/{name}/ navigation.
• Tabs are preloaded, not lazy — Description, History, and Files tabs are all server-rendered in one HTML response. Tab click is CSS-only. Never wait for a network event after clicking a tab. This is the opposite of crates.io.
• Search results show no version number — The .package-snippet elements contain the creation date, not the current version. To get the version, navigate to /project/{name}/ or use the JSON API.
• robots.txt disallows /search* — scraping the search page is against ToS; combined with hCaptcha, the JSON API is the correct path.

Site Map

Page Structures

Homepage

Package Detail (/project/{name}/)

Search Results (/search/?q={query})

Note: Search results do not include the current version number. For current version, navigate to /project/{name}/ or use the JSON API.

Files Tab (within package detail, #files panel)

Navigation Flows

Flow: Search for a Package [Browser]

Starting point: https://pypi.org/ Prerequisites: None

Success state: URL is https://pypi.org/search/?q={term} with .package-snippet results visible.

Branching:

• If button#capSubmit appears → hCaptcha is blocking — search not available for this browser session. Navigate directly to /project/{name}/ if you know the package name, or use the JSON API: GET https://pypi.org/pypi/{name}/json

Flow: View Package Detail [Browser]

Starting point: Any page Prerequisites: None

Success state: Package page with description, version badge, maintainers, and sidebar links visible. Install command visible in header area.

Flow: Browse Version History [Browser]

Starting point: https://pypi.org/project/{name}/ Prerequisites: None

To navigate to a specific version:

Success state: URL becomes https://pypi.org/project/{name}/{version}/ with that version's metadata. Status badge may read "Latest version" or "Yanked".

Flow: Get Package Metadata [API]

Starting point: None — direct HTTP fetch Prerequisites: None

Success state: JSON with top-level keys info, releases, urls, vulnerabilities.

API Response Shapes

GET /pypi/{name}/json

{
  "info": {
    "name": "requests",
    "version": "2.32.3",
    "summary": "Python HTTP for Humans.",
    "requires_python": ">=3.8",
    "home_page": "https://requests.readthedocs.io",
    "license": "Apache-2.0",
    "author": "Kenneth Reitz",
    "author_email": "me@kennethreitz.org",
    "requires_dist": [
      "charset-normalizer<4,>=2",
      "idna<4,>=2.5",
      "urllib3<3,>=1.21.1",
      "certifi>=2017.4.17"
    ],
    "classifiers": [
      "License :: OSI Approved :: Apache Software License",
      "Programming Language :: Python :: 3.11"
    ],
    "project_urls": {
      "Homepage": "https://requests.readthedocs.io",
      "Source": "https://github.com/psf/requests"
    }
  },
  "releases": {
    "2.32.3": [
      {
        "filename": "requests-2.32.3-py3-none-any.whl",
        "url": "https://files.pythonhosted.org/packages/.../requests-2.32.3-py3-none-any.whl",
        "size": 64928,
        "upload_time_iso_8601": "2024-05-29T15:37:05.856790Z",
        "requires_python": ">=3.8",
        "yanked": false,
        "yanked_reason": null
      }
    ]
  },
  "urls": [
    {
      "filename": "requests-2.32.3-py3-none-any.whl",
      "url": "https://files.pythonhosted.org/packages/...",
      "packagetype": "bdist_wheel"
    }
  ],
  "vulnerabilities": []
}

Key fields:

Common Agent Tasks

"What version of {package} should I install?"

1. Fetch: GET https://pypi.org/pypi/{name}/json
2. Read info.version — this is the latest stable version
3. Read info.requires_python — confirm compatibility with user's Python version
4. Read vulnerabilities[] — if non-empty, mention known CVEs
5. Output: pip install {name}=={version} and any compatibility caveats

"Is {package} actively maintained?"

1. Fetch: GET https://pypi.org/pypi/{name}/json
2. Read releases keys — count total releases; sort and get the 3 most recent
3. Read releases[latest][0].upload_time_iso_8601 — when was the last release?
4. Read info.home_page and project_urls.Source — is there a linked repo?
5. Read vulnerabilities[] — any unpatched CVEs?
6. Output: latest release date, total release count, source link, vulnerability count

"Compare {pkg1} and {pkg2}"

1. Fetch both in parallel: GET /pypi/{pkg1}/json and GET /pypi/{pkg2}/json
2. Compare: info.version, info.requires_python, info.license
3. Compare dependency count: len(info.requires_dist)
4. Compare release history: len(releases) and recency of latest release
5. Compare vulnerability status: len(vulnerabilities) for each
6. Output: side-by-side table of version / Python req / license / dep count / release count / last release / vuln count

Dynamic Content Patterns

• MPA with server-rendered HTML: No JS framework. Every page navigation is a full page load. Use domcontentloaded.
• Tabs are already in DOM: All three tab panels (description, history, files) are server-rendered on the same page. Clicking a tab toggles CSS visibility only — no network request, no waiting.
• Version history is fully inline: All releases are listed in the #history panel with no pagination, even for packages with 100+ versions.
• hCaptcha on /search/: Automated headless browsers are frequently challenged by hCaptcha on the search endpoint. Direct package URLs (/project/{name}/) and the JSON API are unaffected.
• Yanked releases: Appear in history with " yanked" in the card text. pip will refuse to install them unless the version is pinned exactly.

Timing & Waiting

Rate Limits

Parallel requests: Safe to fire multiple metadata API requests in parallel — the JSON API is CDN-backed and handles high concurrency.

Common Gotchas

1. hCaptcha blocks automated search — /search/ is blocked for headless browsers. Detect via button#capSubmit. Workaround: use the JSON API (https://pypi.org/pypi/{name}/json) or navigate directly to /project/{name}/ if you know the package name.
2. Tabs don't trigger network requests — Description, history, and files content are all server-rendered in one HTML response. Clicking #history-tab just toggles visibility. Never wait for a network event after tab click.
3. Package name canonicalization — PyPI treats hyphens and underscores as equivalent, and names are case-insensitive. pip_tools, pip-tools, and Pip-Tools all go to the same page. The canonical name is in info.name from the JSON API.
4. Yanked releases still appear in history — Yanked versions stay visible in a.card entries with "yanked" in the text. They won't install via pip without explicit version pinning.
5. robots.txt disallows /search* — Well-behaved crawlers should not scrape search. Combined with hCaptcha, the intended path for programmatic lookup is the JSON API.
6. Version history shows all releases — No pagination on the history tab. document.querySelectorAll('a.card').length gives the total release count.
7. History entries from JSON API — releases key in the JSON API response is an object keyed by version string. Iterate Object.keys(data.releases).reverse() for chronological order.

Anti-Patterns

• NEVER use the browser search page for automation — hCaptcha blocks headless browsers on /search/. Use GET /pypi/{name}/json or navigate directly to /project/{name}/.
• NEVER wait for a network event after clicking a tab — tabs are preloaded in the HTML. Waiting for networkidle or load after a tab click will timeout or proceed immediately with wrong content.
• NEVER expect version number in search results — .package-snippet elements show the creation date (.package-snippet__created), not the current version. There is no current version in search results — go to the detail page or API.
• NEVER use .card__title to extract version — the version text is in a.card > p.release__version (direct child <p>). .card__title does not exist.
• NEVER assume releases is an array — releases is an object keyed by version string, not an array. Iterate Object.keys(data.releases).

Freshness