Shipping huggingface_hub every week with AI, open tools, and a human in the loop

Shipping huggingface_hub every week with AI, open tools, and a human in the loop

The open-source ecosystem for machine learning moves fast. Hugging Face’s huggingface_hub library—the Python client that powers model sharing, dataset downloads, and repository management—receives updates on a nearly weekly cadence. Maintaining that release velocity while ensuring quality, security, and backward compatibility is a non-trivial challenge. Over the past year, the Hugging Face team has refined a workflow that combines AI-assisted code generation, open-source tooling, and a rigorous human review process to ship reliable releases every week. This article unpacks that pipeline, from automated testing to AI-powered PR summaries, and provides a practical guide for teams wanting to adopt similar practices.

Requirements

Before diving into the setup, you’ll need the following components to replicate or adapt the huggingface_hub release workflow:

• Python 3.9+ – The library itself runs on Python, and all automation scripts assume a modern Python environment.
• Git – For version control and CI/CD integration.
• GitHub account – The hub’s development lives on GitHub, leveraging Actions, Issues, and PRs.
• OpenAI API key – Used for generating release notes, PR summaries, and test suggestions. (Alternatively, you can use any compatible LLM API.)
• Hugging Face account – To publish packages to PyPI and manage the huggingface_hub repository.
• Basic familiarity with CI/CD and pytest – The workflow relies on automated testing and continuous integration.

Step-by-Step Installation

1. Clone the Repository and Set Up a Virtual Environment

Start by cloning the official huggingface_hub repository and creating an isolated Python environment.

git clone https://github.com/huggingface/huggingface_hub.git
cd huggingface_hub
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

This ensures that any dependencies you install won’t conflict with system-wide packages.

2. Install Development Dependencies

The project uses poetry for dependency management. Install it and then install all dev dependencies, including testing and linting tools.

pip install poetry
poetry install --with dev

This will pull in pytest, black, ruff, pre-commit, and other tools that enforce code quality.

3. Set Up Pre-commit Hooks

Pre-commit hooks catch formatting and linting issues before code even reaches CI. Run:

pre-commit install

Now every git commit will automatically run checks like trailing-whitespace removal and Python import sorting.

4. Configure the OpenAI API for AI-Assisted Tasks

Create a .env file in the project root to store your API key securely. The release scripts will read from this file.

OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Then install the openai Python client if you haven’t already:

pip install openai

5. Set Up GitHub Actions for Weekly Releases

The actual weekly release is triggered by a GitHub Actions workflow. The relevant YAML file lives at .github/workflows/weekly-release.yml. A minimal version looks like this:

name: Weekly Release
on:
  schedule:
    - cron: '0 10 * * 1'  # Every Monday at 10:00 UTC
  workflow_dispatch:       # Allow manual trigger

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.10'
      - name: Install dependencies
        run: pip install poetry && poetry install
      - name: Run tests
        run: poetry run pytest --cov=huggingface_hub
      - name: Generate release notes
        run: python scripts/generate_release_notes.py
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
      - name: Build and publish
        run: poetry publish --build
        env:
          PYPI_API_TOKEN: ${{ secrets.PYPI_API_TOKEN }}

This workflow runs tests, generates release notes with AI, and publishes to PyPI—all with a single human approval gate before the final publish step.

Usage Examples

Example 1: AI-Generated Release Notes

One of the most time-consuming parts of shipping weekly is writing clear, concise release notes. The team uses an OpenAI-powered script that reads the merged PRs since the last release and produces a draft.

Run the script locally to see how it works:

python scripts/generate_release_notes.py --since v0.23.0 --to v0.24.0

The script queries the GitHub API for all closed PRs between the two tags, then sends a prompt to the LLM asking for a categorized summary. A sample output might look like:

## v0.24.0 - 2024-07-22

### 🚀 New Features
- Add `snapshot_download` resume support (PR #1234)
- Expose `HfApi.get_model_tags` (PR #1245)

### 🐛 Bug Fixes
- Fix `list_repo_files` timeout on large repos (PR #1250)
- Correct type hint for `metadata` parameter (PR #1256)

### 🔧 Improvements
- Reduce memory usage in `huggingface_hub` init (PR #1261)
- Update all HTTP client retry defaults (PR #1270)

The human release manager reviews this draft, edits as needed, and merges it into the release branch.

Example 2: Automated PR Summaries with AI

To speed up code review, every new pull request triggers an AI summarization step. The workflow adds a comment to the PR with a high-level description of the changes, potential risks, and suggested reviewers.

The script behind this uses the OpenAI API to summarize the diff:

import openai
import os
import requests

def summarize_pr(diff_url: str, pr_title: str) -> str:
    # Fetch the diff
    response = requests.get(diff_url)
    diff_text = response.text[:3000]  # Truncate to avoid token limits

    prompt = f"""
    Summarize the following pull request changes for a technical audience.
    PR Title: {pr_title}
    Diff:
    {diff_text}

    Provide:
    - What the PR does
    - Potential breaking changes
    - Suggested review focus areas
    """
    openai.api_key = os.getenv("OPENAI_API_KEY")
    completion = openai.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.3
    )
    return completion.choices[0].message.content

When this script runs in CI, it posts the summary as a PR comment. The human reviewer uses that summary as a starting point, not a replacement for a thorough review.

Example 3: Local Pre-Release Testing with AI-Generated Edge Cases

Before shipping, the team runs a script that uses AI to suggest additional test cases based on recent code changes. This helps catch regressions that standard unit tests might miss.

python scripts/suggest_tests.py --diff HEAD~10

The script extracts the diff of the last 10 commits, sends it to the LLM with a request to propose five edge-case tests, and outputs them as a pytest skeleton:

# Suggested test cases for PR #1280
def test_snapshot_download_partial_failure():
    """Simulate network failure mid-download and verify resume."""
    # ... test body generated by AI

def test_get_model_tags_unicode():
    """Verify that tags with non-ASCII characters are handled."""
    # ... test body generated by AI

The human developer then implements these tests manually, but the suggestions save time and broaden coverage.

The Human in the Loop: Why It Matters

Despite the heavy use of AI, every weekly release includes a mandatory human review step. The AI handles the grunt work—drafting notes, summarizing diffs, suggesting tests—but a human makes the final call on:

• Security implications – AI might not recognize a subtle vulnerability in a new dependency.
• Breaking changes – The LLM may miss API incompatibilities that an experienced maintainer can spot.
• Release timing – The human decides whether to delay a release if a critical bug is uncovered late in the week.

The process is designed so that the AI never pushes code to production or publishes to PyPI without a human click. The GitHub Actions workflow includes an environment approval gate:

deploy:
    runs-on: ubuntu-latest
    environment: production
    needs: [test, generate-notes]
    steps:
      - name: Publish to PyPI
        run: poetry publish

The environment: production setting requires a human to approve the deployment in the GitHub UI before the publish step runs. This simple guardrail ensures that even if the AI-generated notes or tests are flawed, a human catches it before the package reaches users.

Scaling the Workflow to Your Project

The huggingface_hub team’s approach isn’t tied to any single AI provider or CI system. You can adapt it to your own open-source project with a few modifications:

1. Replace the AI model – Swap OpenAI for a local LLM (e.g., via Ollama) if you need to keep data on-premises.
2. Adjust the release cadence – Change the cron schedule from weekly to bi-weekly or monthly.
3. Add more human gates – For high-risk releases, require two approvals instead of one.
4. Use the same scripts – The generate_release_notes.py and suggest_tests.py scripts are open source and can be forked from the huggingface_hub repository.

Conclusion

Shipping huggingface_hub every week is a testament to what’s possible when you combine AI’s speed with human judgment. The AI handles the repetitive, time-consuming tasks—writing release notes, summarizing PRs, suggesting tests—while humans focus on the high-level decisions that require context, intuition, and accountability. The result is a steady, reliable release cadence that has helped the library grow into one of the most widely used tools in the ML ecosystem.

For teams looking to adopt a similar workflow, the key takeaway is simple: let the AI do the drafting, but keep a human in the loop for the final sign-off. That balance of automation and oversight is what makes shipping every week not just possible, but sustainable.