Job Scout: A LangGraph Agent That Actually Matches CVs to Jobs

July 26, 2026 • python langgraph anthropic observability agents

I built an AI agent that reads a CV, extracts a structured profile, searches multiple job boards, and scores every result with an honest 0–100 fit rating and a plain-English explanation. It runs locally, costs pennies per search, and every LLM call is traced. Here’s how the whole thing works.

CV (PDF)
  │
  ▼
cv_reader ──▶ profile.py (LLM → structured Profile)
                 │
                 ▼
              runner.py ──▶ LangGraph StateGraph
                              │
                    ┌─────────┼─────────┐
                    ▼         ▼         ▼
               fetch_jobs  rank_jobs  reformulate_query
                    │         │         │
                    └─────────┼─────────┘
                              ▼
                        Ranked Jobs ──▶ SQLite (history)
                              │
                              ▼
                         Gradio UI

Why I built it

Job boards are noisy. You search “ML Engineer,” get 200 results, and spend hours reading descriptions to figure out which ones actually match your background. I wanted to flip that — hand the machine a CV, let it search, and get back a ranked shortlist with explanations I could trust.

The real motivation was to learn LangGraph properly. Not a toy chatbot — a multi-step agent with tool calls, structured output, conditional loops, and a budget cap. Something where the graph topology actually matters.

The agent graph

The core is a LangGraph StateGraph with three nodes and a conditional loop:

fetch_jobs — The LLM receives the candidate profile and picks search arguments via a tool call. It calls run_search(), which cascades through available job sources. Results are deduplicated against jobs found in prior loops so the agent doesn’t re-rank the same posting twice.

rank_jobs — Jobs are batched in groups of 5. For each batch, the LLM scores every job using structured output (with_structured_output), producing a RankedJob with a 0–100 fit score, a plain-text explanation, matched skills, and identified gaps. Results are sorted by fit descending.

reformulate_query — If fewer than 5 jobs scored 60 or above, and the agent hasn’t already reformulated twice, it broadens the search query and loops back to fetch_jobs. This is the part that makes it an agent rather than a pipeline — it reacts to its own results.

The loop exits when enough good matches are found or the reformulation cap (2) is hit. A hard budget of 25 LLM calls per run (ensure_budget()) acts as a circuit breaker so a bad prompt can’t run up the bill.

Job source cascade

Not every user has API keys, and I didn’t want the app to be unusable without them. The search function tries sources in priority order and falls back gracefully:

  1. JSearch — Google-for-Jobs aggregator. Best location filtering. Requires an API key (~200 free requests/month).
  2. Adzuna — International coverage, free registration. Requires app ID + key.
  3. Remotive — Remote-only jobs. No key needed.
  4. Offline cache — A committed JSON file with ~247 jobs. Always works.

Each source implements a JobSource protocol and returns normalized JobPosting objects. The cascade means the app runs with zero API keys out of the box — you just get a narrower job pool.

Structured output everywhere

The agent relies heavily on Pydantic models as structured-output targets:

  • Profile — name, seniority, primary roles, skills, years of experience, locations, languages, remote preference. Extracted from raw CV text in one LLM call.
  • JobScore — fit score (0–100), explanation, matched skills, gaps. One per job, batched 5 at a time.
  • RankedJob — a JobPosting paired with its score. This is what the user sees.

Structured output is where model quality matters most. Local models (I tried Ollama with Qwen 2.5 7B) struggled with the schema — they’d return malformed JSON, skip required fields, or hallucinate fit scores that didn’t match their own explanations. Switching to Claude Sonnet 4.5 via the Anthropic API made the output reliable. The scores became consistent, the explanations got specific, and the matched-skills lists actually reflected what was in the CV.

The LLM factory

The get_chat_model() function wraps LangChain’s init_chat_model with provider-aware key export. It accepts a provider:model string and handles the rest:

# config.py default
SCOUT_MODEL=anthropic:claude-sonnet-4-5-20250929

# also works
SCOUT_MODEL=openai:gpt-4o-mini
SCOUT_MODEL=ollama:llama3.2
SCOUT_MODEL=groq:llama-3.3-70b-versatile

Models are cached with lru_cache so the same model string doesn’t create multiple clients. The provider prefix determines which API key gets exported to the environment (LangChain’s clients read from os.environ, not from the settings object).

Persistence

The app was originally stateless — results vanished when you closed the tab. I added a SQLite layer so the History tab could browse past searches across restarts. Five tables:

  • cvs — raw CV text and filename
  • profiles — structured profile linked to a CV
  • runs — metadata (model, cost, latency, reformulation count, errors)
  • job_postings — deduplicated across runs via INSERT OR IGNORE
  • run_jobs — junction table with per-run fit scores and explanations

List fields (skills, tags, gaps) are stored as JSON text columns. Booleans are INTEGER 0/1. Timestamps are ISO 8601. WAL mode is enabled for concurrent reads from the Gradio UI.

All writes are best-effort — if the database is unavailable, the search still works. The persistence layer logs warnings and moves on.

What I learned

Structured output is the bottleneck. The agent graph topology is straightforward. The hard part is getting the LLM to reliably fill a Pydantic schema with accurate, calibrated scores. Small models can’t do it well. Claude Sonnet 4.5 can.

Cascading fallbacks make demos work. Nothing kills a demo faster than “you need three API keys to try this.” The source cascade means make app works immediately, even offline.

Budget caps are essential. Without ensure_budget(), a reformulation loop with a vague CV could make dozens of LLM calls. The 25-call cap turns a potential $2 run into a predictable $0.15 one.

SQLite is underrated for single-user apps. WAL mode, JSON columns, and INSERT OR IGNORE give you 80% of what you’d want from Postgres with zero infrastructure.

The stack

  • LangGraph — agent graph with StateGraph and MemorySaver checkpointer
  • Anthropic Claude Sonnet 4.5 — default LLM (swappable via env var)
  • LangChain — model abstraction, structured output, tool calling
  • Gradio — 3-step wizard UI with History tab
  • SQLite — persistence with WAL mode
  • Pydantic — data models and structured-output targets
  • pydantic-settings — configuration from .env with SecretStr
  • httpx — async HTTP for job source APIs
  • Prometheus — /metrics endpoint for homelab monitoring
  • Ruff — linting and formatting
  • pytest + respx — offline test suite (59 tests, no network, no credentials)

The code is at github.com/dirtmerchant/observable-job-agent.

← All posts