Shipping huggingface_hub every week with AI, open tools, and a human in the loop

Shipping huggingface_hub every week with AI, open tools, and a human in the loop

The open-source ecosystem for machine learning moves fast. Hugging Face’s `huggingface_hub` library—the Python client that powers model sharing, dataset downloads, and repository management—receives updates on a nearly weekly cadence. Maintaining that release velocity while ensuring quality, security, and backward compatibility is a non-trivial challenge. Over the past year, the Hugging Face team has refined a workflow that combines AI-assisted code generation, open-source tooling, and a rigorous human review process to ship reliable releases every week. This article unpacks that pipeline, from automated testing to AI-powered PR summaries, and provides a practical guide for teams wanting to adopt similar practices.

Requirements

Before diving into the setup, you’ll need the following components to replicate or adapt the `huggingface_hub` release workflow:

• **Python 3.9+** – The library itself runs on Python, and all automation scripts assume a modern Python environment.
• **Git** – For version control and CI/CD integration.
• **GitHub account** – The hub’s development lives on GitHub, leveraging Actions, Issues, and PRs.
• **OpenAI API key** – Used for generating release notes, PR summaries, and test suggestions. (Alternatively, you can use any compatible LLM API.)
• **Hugging Face account** – To publish packages to PyPI and manage the `huggingface_hub` repository.
• **Basic familiarity with CI/CD and pytest** – The workflow relies on automated testing and continuous integration.

Step-by-Step Installation

1. Clone the Repository and Set Up a Virtual Environment

Start by cloning the official `huggingface_hub` repository and creating an isolated Python environment.

git clone https://github.com/huggingface/huggingface_hub.git
cd huggingface_hub
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

This ensures that any dependencies you install won’t conflict with system-wide packages.

2. Install Development Dependencies

The project uses `poetry` for dependency management. Install it and then install all dev dependencies, including testing and linting tools.

pip install poetry
poetry install --with dev

This will pull in `pytest`, `black`, `ruff`, `pre-commit`, and other tools that enforce code quality.

3. Set Up Pre-commit Hooks

Pre-commit hooks catch formatting and linting issues before code even reaches CI. Run:

pre-commit install

Now every `git commit` will automatically run checks like trailing-whitespace removal and Python import sorting.

4. Configure the OpenAI API for AI-Assisted Tasks

Create a `.env` file in the project root to store your API key securely. The release scripts will read from this file.

OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Then install the `openai` Python client if you haven’t already:

pip install openai

5. Set Up GitHub Actions for Weekly Releases

The actual weekly release is triggered by a GitHub Actions workflow. The relevant YAML file lives at `.github/workflows/weekly-release.yml`. A minimal version looks like this:

name: Weekly Release
on:
  schedule:
    - cron: '0 10 * * 1'  # Every Monday at 10:00 UTC
  workflow_dispatch:       # Allow manual trigger

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.10'
      - name: Install dependencies
        run: pip install poetry && poetry install
      - name: Run tests
        run: poetry run pytest --cov=huggingface_hub
      - name: Generate release notes
        run: python scripts/generate_release_notes.py
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
      - name: Build and publish
        run: poetry publish --build
        env:
          PYPI_API_TOKEN: ${{ secrets.PYPI_API_TOKEN }}

This workflow runs tests, generates release notes with AI, and publishes to PyPI—all with a single human approval gate before the final publish step.

Usage Examples

Example 1: AI-Generated Release Notes

One of the most time-consuming parts of shipping weekly is writing clear, concise release notes. The team uses an OpenAI-powered script that reads the merged PRs since the last release and produces a draft.

Run the script locally to see how it works:

python scripts/generate_release_notes.py --since v0.23.0 --to v0.24.0

The script queries the GitHub API for all closed PRs between the two tags, then sends a prompt to the LLM asking for a categorized summary. A sample output might look like:

## v0.24.0 - 2024-07-22

### 🚀 New Features
- Add `snapshot_download` resume support (PR #1234)
- Expose `HfApi.get_model_tags` (PR #1245)

### 🐛 Bug Fixes
- Fix `list_repo_files` timeout on large repos (PR #1250)
- Correct type hint for `metadata` parameter (PR #1256)

### 🔧 Improvements
- Reduce memory usage in `huggingface_hub` init (PR #1261)
- Update all HTTP client retry defaults (PR #1270)

The human release manager reviews this draft, edits as needed, and merges it into the release branch.

Example 2: Automated PR Summaries with AI

To speed up code review, every new pull request triggers an AI summarization step. The workflow adds a comment to the PR with a high-level description of the changes, potential risks, and suggested reviewers.

The script behind this uses the OpenAI API to summarize the diff:

import openai
import os
import requests

def summarize_pr(diff_url: str, pr_title: str) -> str:
    # Fetch the diff
    response = requests.get(diff_url)
    diff_text = response.text[:3000]  # Truncate to avoid token limits

    prompt = f"""
    Summarize the following pull request changes for a technical audience.
    PR Title: {pr_title}
    Diff:
    {diff_text}

    Provide:
    - What the PR does
    - Potential breaking changes
    - Suggested review focus areas
    """
    openai.api_key = os.getenv("OPENAI_API_KEY")
    completion = openai.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.3
    )
    return completion.choices[0].message.content

When this script runs in CI, it posts the summary as a PR comment. The human reviewer uses that summary as a starting point, not a replacement for a thorough review.

Example 3: Local Pre-Release Testing with AI-Generated Edge Cases

Before shipping, the team runs a script that uses AI to suggest additional test cases based on recent code changes. This helps catch regressions that standard unit tests might miss.

python scripts/suggest_tests.py --diff HEAD~10

The script extracts the diff of the last 10 commits, sends it to the LLM with a request to propose five edge-case tests, and outputs them as a pytest skeleton:

# Suggested test cases for PR #1280
def test_snapshot_download_partial_failure():
    """Simulate network failure mid-download and verify resume."""
    # ... test body generated by AI

def test_get_model_tags_unicode():
    """Verify that tags with non-ASCII characters are handled."""
    # ... test body generated by AI

The human developer then implements these tests manually, but the suggestions save time and broaden coverage.

The Human in the Loop: Why It Matters

Despite the heavy use of AI, every weekly release includes a mandatory human review step. The AI handles the grunt work—drafting notes, summarizing diffs, suggesting tests—but a human makes the final call on:

• **Security implications** – AI might not recognize a subtle vulnerability in a new dependency.
• **Breaking changes** – The LLM may miss API incompatibilities that an experienced maintainer can spot.
• **Release timing** – The human decides whether to delay a release if a critical bug is uncovered late in the week.

The process is designed so that the AI never pushes code to production or publishes to PyPI without a human click. The GitHub Actions workflow includes an `environment` approval gate:

deploy:
    runs-on: ubuntu-latest
    environment: production
    needs: [test, generate-notes]
    steps:
      - name: Publish to PyPI
        run: poetry publish

The `environment: production` setting requires a human to approve the deployment in the GitHub UI before the publish step runs. This simple guardrail ensures that even if the AI-generated notes or tests are flawed, a human catches it before the package reaches users.

Scaling the Workflow to Your Project

The `huggingface_hub` team’s approach isn’t tied to any single AI provider or CI system. You can adapt it to your own open-source project with a few modifications:

1. **Replace the AI model** – Swap OpenAI for a local LLM (e.g., via Ollama) if you need to keep data on-premises. 2. **Adjust the release cadence** – Change the cron schedule from weekly to bi-weekly or monthly. 3. **Add more human gates** – For high-risk releases, require two approvals instead of one. 4. **Use the same scripts** – The `generate_release_notes.py` and `suggest_tests.py` scripts are open source and can be forked from the `huggingface_hub` repository.

Conclusion

Shipping `huggingface_hub` every week is a testament to what’s possible when you combine AI’s speed with human judgment. The AI handles the repetitive, time-consuming tasks—writing release notes, summarizing PRs, suggesting tests—while humans focus on the high-level decisions that require context, intuition, and accountability. The result is a steady, reliable release cadence that has helped the library grow into one of the most widely used tools in the ML ecosystem.

For teams looking to adopt a similar workflow, the key takeaway is simple: let the AI do the drafting, but keep a human in the loop for the final sign-off. That balance of automation and oversight is what makes shipping every week not just possible, but sustainable.