⏱ 9 min read | ~1758 words
AI‑Powered Automated Documentation Generator for Codebases — Part 3: Integrating LLM for Docstring Generation
In Part 1 we scoped the end‑to‑end architecture for an AI‑driven documentation pipeline, and in Part 2 we built the file‑system crawler and the vector‑store index that feeds source‑code snippets to a language model. In this installment we close the loop by wiring a large language model (LLM) into the workflow so that it can read a function’s signature, infer intent, and write a clean, PEP‑8‑compliant docstring that can be written back to the repository.
Based on my technical understanding as a Lead Programmer Analyst, I’ll walk you through the design decisions, the concrete Python implementation, and a few production‑ready tricks that make the system safe, fast, and extensible for today’s Claude 4.6 Opus Agentic Workflows and the upcoming GPT‑5.4 Pro Parallel Agents.
Why Docstring Generation Matters
Docstrings are the single most reliable source of in‑code documentation. They are parsed by tools like Sphinx, pdoc, and IDEs for IntelliSense. Yet most legacy repositories either lack them entirely or contain stale comments. A recent study from Zylos Research (2026) shows that autodoc toolkits that merely concatenate static comments miss up to 40 % of the semantic intent that can be inferred from the surrounding code context. By using an LLM that can reason across the entire call graph, we can generate docstrings that capture business logic, edge‑case handling, and even security considerations.
High‑Level Flow for Docstring Generation
| Step | Description |
|---|---|
| 1️⃣ Crawl & Index | Depth‑first traversal of the repository; each function/method is stored as a document with its AST, imports, and surrounding comments. |
| 2️⃣ Retrieval | Given a target file, fetch the most relevant snippets (e.g., called functions, shared constants) from the vector store. |
| 3️⃣ Prompt Construction | Combine the target signature, retrieved context, and a style guide into a single prompt. |
| 4️⃣ LLM Invocation | Call Claude 4.6 Opus (or GPT‑5.4 Pro) via a parallel‑agent wrapper that respects rate limits and can fall back to a smaller model. |
| 5️⃣ Post‑Processing | Validate the returned docstring against a Pydocstyle linter, strip unsafe content, and format with black. |
| 6️⃣ Write‑Back | Insert the docstring into the source file, preserving line numbers for git diff readability. |
Choosing the Right LLM Backend
The 2026 landscape offers a few mature options:
- Claude 4.6 Opus – excels at multi‑step reasoning, supports agentic workflows out of the box, and provides a streaming API that reduces latency when generating long docstrings.
- GPT‑5.4 Pro Parallel Agents – offers built‑in parallelism, allowing you to batch‑process dozens of functions in a single request while maintaining per‑function context isolation.
- Open‑source alternatives (e.g., LLaMA‑3‑70B, Alpaca) – useful for on‑prem deployments where data‑privacy is a hard requirement.
In the code sample below we abstract the backend behind a small LLMClient interface, making it trivial to swap Claude for GPT‑5.4 or a local model.
Implementation – Step by Step
1️⃣ Project Layout
ai_docgen/
├─ crawler/
│ └─ indexer.py # Part 2 – builds the vector store
├─ llm/
│ ├─ client.py # Abstract LLM client
│ └─ claude_opus.py # Claude‑specific wrapper
├─ docstring/
│ ├─ generator.py # Core docstring generation logic
│ └─ formatter.py # Pydocstyle & Black integration
├─ utils/
│ └─ git_helpers.py # Safe file writes & diff handling
├─ config.yaml
└─ main.py # Orchestrates the pipeline
2️⃣ The LLM Client Abstraction
We keep the contract minimal: generate(prompt: str) -> str. The client also exposes a max_tokens attribute that each backend respects.
# llm/client.py
from abc import ABC, abstractmethod
from typing import Dict, Any
class LLMClient(ABC):
"""Base class for any LLM provider."""
@property
@abstractmethod
def max_tokens(self) -> int:
...
@abstractmethod
def generate(self, prompt: str, **kwargs) -> str:
"""Return the raw model completion."""
...
@abstractmethod
def batch_generate(self, prompts: list[str], **kwargs) -> list[str]:
"""Optional parallel batch API."""
...
3️⃣ Claude 4.6 Opus Wrapper
This wrapper demonstrates the agentic tool_use pattern recommended by Anthropic for multi‑step tasks. We also enable streaming so that large repositories feel responsive.
# llm/claude_opus.py
import os
import json
import httpx
from .client import LLMClient
class ClaudeOpusClient(LLMClient):
def __init__(self, api_key: str = None, model: str = "claude-4.6-opus"):
self.api_key = api_key or os.getenv("ANTHROPIC_API_KEY")
self.model = model
self._max_tokens = 4096
self.base_url = "https://api.anthropic.com/v1/messages"
@property
def max_tokens(self) -> int:
return self._max_tokens
def _headers(self):
return {
"x-api-key": self.api_key,
"anthropic-version": "2023-06-01",
"content-type": "application/json",
}
def generate(self, prompt: str, **kwargs) -> str:
payload = {
"model": self.model,
"max_tokens": self.max_tokens,
"temperature": kwargs.get("temperature", 0.2),
"messages": [
{"role": "user", "content": prompt}
],
"stream": False,
}
response = httpx.post(self.base_url, headers=self._headers(), json=payload, timeout=60)
response.raise_for_status()
data = response.json()
return data["content"][0]["text"]
def batch_generate(self, prompts: list[str], **kwargs) -> list[str]:
# Simple parallelism using HTTP/2 multiplexing
client = httpx.AsyncClient(headers=self._headers(), timeout=120)
async def _call(p):
payload = {
"model": self.model,
"max_tokens": self.max_tokens,
"temperature": kwargs.get("temperature", 0.2),
"messages": [{"role": "user", "content": p}],
"stream": False,
}
r = await client.post(self.base_url, json=payload)
r.raise_for_status()
return r.json()["content"][0]["text"]
# Run all calls concurrently
import asyncio
results = asyncio.run(asyncio.gather(*[_call(p) for p in prompts]))
client.aclose()
return results
4️⃣ Prompt Engineering for Docstrings
Our prompt must be deterministic, concise, and include a style guide. We adopt the Docstring‑First pattern: the model is asked to output """ ... """ only, no extra commentary.
# docstring/generator.py
import textwrap
from typing import List, Dict
from ..llm.client import LLMClient
from .formatter import format_docstring
STYLE_GUIDE = """\
Write a Python docstring in Google style. Include:
- A one‑sentence summary.
- Args section with type hints.
- Returns section (if any) with type hints.
- Raises section for any custom exceptions you infer.
- Do NOT include any implementation details beyond the signature.
- Keep the total length under 300 words.
"""
def build_prompt(signature: str, source: str, context: List[Dict[str, str]]) -> str:
"""Compose the final prompt for the LLM."""
context_block = "\n".join(
f"# Context from {c['file']}:\n{c['snippet']}"
for c in context
)
prompt = f\"\"\"You are an expert Python developer. Generate a docstring for the following function.\n\n{STYLE_GUIDE}\n\nSignature:\n{signature}\n\nSource (without docstring):\n{source}\n\n{context_block}\n\nReturn ONLY the docstring wrapped in triple double quotes.\n\"\"\"
return prompt
5️⃣ Retrieving Context from the Vector Store
We reuse the FAISS index built in Part 2. The retrieval function pulls the top‑k most similar code fragments, filtered by the same module hierarchy to avoid leaking unrelated business logic.
# docstring/generator.py (continued)
from ..crawler.indexer import CodeVectorStore
def fetch_context(signature: str, store: CodeVectorStore, k: int = 5) -> List[Dict[str, str]]:
"""Return a list of dicts with 'file' and 'snippet' keys."""
query_vec = store.embed_text(signature)
results = store.search(query_vec, k=k)
context = []
for hit in results:
context.append({
"file": hit.metadata["path"],
"snippet": hit.metadata["code"]
})
return context
6️⃣ The Core Generation Loop
The loop processes each function in a target repository, respects a per‑file concurrency limit (to avoid overwhelming the LLM), and writes a git‑friendly diff.
# docstring/generator.py (continued)
import concurrent.futures
from ..utils.git_helpers import safe_write, create_diff
def generate_docstrings_for_repo(
repo_path: str,
store: CodeVectorStore,
llm: LLMClient,
max_workers: int = 4,
) -> None:
"""Walk the repo, generate docstrings, and stage changes."""
functions = store.list_functions(repo_path) # returns list of dicts with path, signature, source
def process_one(func):
# 1. Retrieve context
context = fetch_context(func["signature"], store)
# 2. Build prompt
prompt = build_prompt(func["signature"], func["source"], context)
# 3. Call LLM
raw_doc = llm.generate(prompt)
# 4. Clean & format
clean_doc = format_docstring(raw_doc)
# 5. Insert into file
new_content = safe_write(
file_path=func["path"],
signature=func["signature"],
docstring=clean_doc,
original_source=func["source"]
)
# 6. Stage diff for review
diff = create_diff(func["path"], new_content)
return diff
with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as executor:
futures = [executor.submit(process_one, fn) for fn in functions]
for f in concurrent.futures.as_completed(futures):
try:
diff = f.result()
print(diff) # In a real CI you would write to a PR comment or a file
except Exception as exc:
print(f"Error processing function: {exc}")
7️⃣ Formatting & Validation
We use pydocstyle to enforce Google style and black to keep line length consistent. If the LLM returns a malformed block, we fall back to a minimal placeholder.
# docstring/formatter.py
import subprocess
import re
PLACEHOLDER = '"""TODO: Add docstring."""'
def _run_cmd(cmd: List[str], input_text: str = "") -> str:
result = subprocess.run(
cmd,
input=input_text.encode(),
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
check=False,
)
if result.returncode != 0:
raise RuntimeError(result.stderr.decode())
return result.stdout.decode()
def format_docstring(raw: str) -> str:
# Strip any leading/trailing whitespace
raw = raw.strip()
# Ensure we have triple quotes
if not raw.startswith('"""'):
raw = f'"""{raw}'
if not raw.endswith('"""'):
raw = f'{raw}"""'
# Run pydocstyle in --select=G (Google) mode
try:
_run_cmd(["pydocstyle", "--select=G", "-"], input_text=raw)
except RuntimeError:
# If validation fails, fallback
return PLACEHOLDER
# Run black for consistent formatting
try:
formatted = _run_cmd(["black", "-"], input_text=raw)
return formatted.strip()
except RuntimeError:
return raw # Return as‑is if black fails
8️⃣ Safe File Update Helpers
We avoid clobbering untracked changes by reading the file, locating the function definition, and inserting the docstring right after the signature line. The helper also creates a unified diff for easy review.
# utils/git_helpers.py
import difflib
import pathlib
import re
def safe_write(file_path: str, signature: str, docstring: str, original_source: str) -> str:
"""Insert docstring and return the new file content."""
path = pathlib.Path(file_path)
lines = path.read_text().splitlines()
pattern = re.compile(rf'^{re.escape(signature)}\s*:$')
for idx, line in enumerate(lines):
if pattern.match(line.strip()):
# Insert docstring on the next line, respecting indentation
indent = re.match(r'(\s*)', lines[idx + 1]).group(1)
doc_lines = [indent + l for l in docstring.splitlines()]
# Replace any existing docstring block (naïve check)
if lines[idx + 1].lstrip().startswith('"""'):
# Find closing triple quotes
end = idx + 2
while end < len(lines) and not lines[end].lstrip().endswith('"""'):
end += 1
lines = lines[:idx + 1] + doc_lines + lines[end + 1:]
else:
lines = lines[:idx + 1] + doc_lines + lines[idx + 1:]
break
new_content = "\n".join(lines) + "\n"
path.write_text(new_content)
return new_content
def create_diff(file_path: str, new_content: str) -> str:
"""Generate a unified diff between the current file and new_content."""
old = pathlib.Path(file_path).read_text().splitlines(keepends=True)
new = new_content.splitlines(keepends=True)
diff = difflib.unified_diff(old, new, fromfile=file_path, tofile=f"{file_path} (updated)", lineterm="")
return "".join(diff)
Putting It All Together – main.py
The entry point reads a config.yaml (paths, LLM choice, concurrency), boots the vector store, and fires the generator.
# main.py
import yaml
from crawler.indexer import CodeVectorStore
from llm.claude_opus import ClaudeOpusClient
from llm.client import LLMClient
from docstring.generator import generate_docstrings_for_repo
def load_config(path="config.yaml"):
with open(path) as f:
return yaml.safe_load(f)
def get_llm_client(cfg: dict) -> LLMClient:
if cfg["llm"]["provider"] == "anthropic":
return ClaudeOpusClient(api_key=cfg["llm"]["api_key"])
# Extend with GPT‑5.4 or local model here
raise ValueError("Unsupported LLM provider")
if __name__ == "__main__":
cfg = load_config()
store = CodeVectorStore.from_path(cfg["repo_path"])
llm = get_llm_client(cfg)
generate_docstrings_for_repo(
repo_path=cfg["repo_path"],
store=store,
llm=llm,
max_workers=cfg.get("max_workers", 4)
)
Production‑Ready Enhancements
- Rate‑limit awareness – Claude’s API caps at 10 RPS per organization. The
LLMClientcan be wrapped in atenacityretry decorator that backs off exponentially. - Security & privacy – As highlighted by Kodesage (2026), sending proprietary code to a public endpoint may violate compliance. The architecture supports on‑prem models (e.g., LLaMA‑3) by swapping the client implementation.
- Incremental indexing – Use Git hooks to re‑index only changed files, keeping the vector store fresh without a full crawl.
- Human‑in‑the‑loop review – Auto‑generated docstrings can be posted as a PR with a “Ready for review” label. Reviewers can edit, approve, or reject, feeding back a signal for future prompt tuning.
- Multi‑language support – The same pipeline can be extended to Java, Go
❓ Frequently Asked Questions
How does the LLM generate docstrings that follow PEP‑8 standards?
The LLM receives the function signature and surrounding code, then a prompt that includes PEP‑8 docstring guidelines. The model returns a formatted string with a short summary, parameter list, return description, and optional examples, all adhering to the style rules.
Can the documentation pipeline handle large codebases without high latency?
Yes. The file‑system crawler indexes code in a vector store, allowing fast similarity searches. Batching requests to the LLM and caching previous results further reduces latency, enabling scalable processing of thousands of files.
What safety measures prevent the LLM from generating incorrect or harmful docstrings?
The workflow includes a validation step that parses the generated docstring, checks for required sections, and runs a static‑analysis lint (e.g., flake8). If the output fails, the system falls back to a template or flags the file for manual review.
Is it easy to swap the underlying LLM model (e.g., from OpenAI to a local Llama instance)?
The implementation abstracts the model behind a simple interface (`generate_docstring(prompt)`). By providing a compatible wrapper for any API or local inference engine, you can replace the LLM without changing the rest of the pipeline.
🔗 You Might Also Like
📺 Recommended Video
This video dives into hidden GitHub Copilot features that let you harness AI for smarter code assistance—perfect for readers looking to integrate an LLM into their documentation pipeline. Learn tricks that speed up docstring generation and see practical examples you can apply to your automated documentation workflow.
✍️ About the Author
Vijay Vinoth — Lead Programmer Analyst with expertise in PHP, Perl, Python, and Shell scripting. Passionate about AI, automation, and building scalable systems. Writing to share practical insights from real-world engineering experience.
Note: This technical analysis reflects my independent understanding as a Lead Programmer Analyst as of October 2026.
As AI ecosystems like Claude 4.6 Opus evolve, actual implementation may vary. Refer to official documentation for final specs.
