QuestGen.
From course evidence to a reviewed exam. ↘A course-grounded academic workspace with human-controlled assessment generation.
QuestGen is a Next.js 16 and FastAPI application for professors with three connected modes: cited course-material chat, an interactive research-and-question-drafting agent, and an exam-paper builder. A local SQLite workspace stores courses, indexed documents, conversations, plans, question drafts and review history. Source restrictions, approval checkpoints, batch generation and printable PDF/DOCX exports are enforced by backend services rather than entrusted to the language model.
The problem, and the approach.
What needed solving?
A useful academic assessment must respect the exact course materials, cover the intended topics, vary cognitive demand across different question formats, and preserve the professor’s ability to change or reject a draft. Sending a whole syllabus to one unconstrained prompt makes source boundaries, marks distribution, question quality and revisions difficult to control.
What was built?
Separate open-ended academic assistance from question consultation and final paper construction. In Chat Mode, retrieve cited course excerpts. In Agent Mode, discuss requirements, optionally research, propose a versioned plan and wait for explicit approval before drafting editable questions. In Exam Mode, validate the blueprint and selected documents, infer slot-level complexity when unspecified, draft question sets in bounded batches, and let the professor keep, revise, reorder and finally approve the paper for export.
Architecture, without the noise.
This is a simplified conceptual diagram of the components documented in the repository; it is not a claim that all operations use a single synchronous pipeline.
Upload PDF, PPTX, DOCX, text and image files; native extraction plus optional model-assisted visual reading.
SQLite stores courses, threads, chunks and versions; FTS5/BM25 search with optional Ollama dense-embedding fusion.
Cited Q&A; tool-driven consultation and editable drafts; approved blueprint and source-scoped paper building.
Agent plan approvals or Exam Mode slot planning; context-engineered, mixed-format batch generation and validation.
Select, edit, regenerate, accept, reorder or reject questions; keep versions and require final approval.
FastAPI exports student or instructor versions as PDF or DOCX only after paper approval.
How it works, step by step.
- 01
Ingest and index the course
Create a course, upload supported material and extract ordered content. PDF pages and presentation slides can receive optional visual transcription. Persist chunks and searchable metadata in SQLite; report coverage and warnings when visual analysis is incomplete.
backend/ingestion.py - 02
Retrieve under the chosen source scope
Use course-scoped SQLite FTS5/BM25 search, optionally fused with local Ollama embeddings. When files are explicitly selected, pass the allowed document IDs into retrieval and exam context preparation instead of searching the whole library.
backend/documents.py - 03
Consult in Agent Mode
Discuss educational goals and question mix rather than applying a fixed MCQ checklist. A streaming tool-calling agent can inspect course material and perform bounded web research, then save a versioned question plan that requires an explicit UI approval.
backend/agent_mode.py - 04
Draft and revise questions
After approved consultation, store complete question drafts with answers and explanations. The professor can edit, revise, select and carry chosen questions into Exam Mode without starting a second agent drafting pipeline.
backend/agent_drafts.py - 05
Plan and draft a full paper in Exam Mode
Validate section counts, marks and source scope. Build slot-specific evidence and infer difficulty/Bloom demand when unspecified; optional Deep Agents planning remains advisory. Generate mixed-format questions in batches, splitting invalid or oversized batches for retries.
backend/exam_batch.py - 06
Review, finalize and export
Keep or revise selected drafts, maintain question versions and ordering, then explicitly approve the complete paper. Student and instructor exports are available as PDF or DOCX.
backend/exam_builder.py
The choices behind the system.
Three different jobs, not five legacy agents
Chat Mode answers and cites materials, Agent Mode handles consultation/research/drafts, and Exam Mode handles validated assessment structure and printing. The old Extractor → Creator → Analysis → Decider → Formatter chain is not this codebase’s architecture.
Human approval belongs to FastAPI
A model can suggest a plan, topic coverage, question content or a revision. Only explicit requests from the professor can approve a consultation plan, accept paper drafts, or authorize final export.
One context-engineered batch, then selective revisions
Exam Mode groups mixed question formats into larger generation requests, checks slot IDs and source citations, splits malformed batches when necessary, and preserves already accepted questions instead of invoking full reasoning separately for each item.
Complexity is inferred, not demanded
When a professor leaves difficulty on auto, the planning baseline considers marks, question type and assessment purpose. An optional Deep Agents planner may enrich topics and Bloom levels but cannot override explicitly set difficulty or change the approved blueprint.
Inside the implementation.
Source grounding, consultation and approval
Hybrid retrieval with a lexical fallback
SQLite FTS5/BM25 runs locally. If QUESTGEN_EMBED_MODEL is configured, Ollama embeddings add a dense candidate set with reciprocal-rank fusion; lexical retrieval remains available without embeddings.
Multimodal lecture materials
PDF, DOCX, PPTX, TXT, Markdown and common image formats are accepted. Native extraction is preserved if optional vision transcription or slide rendering cannot verify diagrams and scanned content.
Consultation and research boundaries
Agent Mode’s tool-calling loop supports course search, up to two web searches and three page reads per turn, versioned plan proposals, approved draft creation and individual question revisions.
Exam Mode invariants
The professor approves exact marks/format sections and selects all-course, selected-file or general-knowledge scope. FastAPI validates slot counts, uniqueness, citations and versions, with a final paper signature for export.
Configurable model access
Backend provider connections include OpenAI, Anthropic, OpenRouter, 9router and Ollama; stored credentials are encrypted locally. This is a configurable model API, not a claimed fully offline product.
Enforcing source-scoped exam evidence
Excerpt from the uploaded implementation. Exam Mode retrieves against the approved document IDs and rejects evidence outside that set before question drafting.
found = await asyncio.to_thread(
documents.retrieve, session['course_id'], section['topic'], 12, material_ids)
if not found:
found = await asyncio.to_thread(
documents.sample_course_sources, session['course_id'], 12, material_ids)
if not found:
raise ValueError(f'No indexed evidence for section {section["id"]}. Reprocess the course files.')
if any(s.get('document_id') and s['document_id'] not in material_ids for s in found):
raise ValueError('Retrieval returned an unapproved document')SOURCE REFERENCE / backend/exam_context.py · build()Results in context.
Implementation evidence, not an accuracy benchmark
The uploaded code contains FastAPI and frontend tests for consultation approval, Agent-to-Exam handoff, source restrictions, ingestion resilience, asynchronous batch generation, question editing/versioning, and paper export. These automated tests check behavior and constraints, not independent learning outcomes or model-generated question correctness. The source archive does not establish a measured hallucination rate, teacher time saved, production uptime or human-rated assessment quality; no such metric is claimed here.
Numbers here are reported by the project author in the linked public repository or CV. They are not independent tests performed for this portfolio.
Trade-offs & open questions.
Single-user local security model
The FastAPI service explicitly does not implement multi-user authentication. Course separation is logical; it must not be exposed publicly as a multi-tenant service without access controls.
Question accuracy still requires professor review
Model-produced answers, complexity assignments and citations can be wrong or pedagogically weak. Review controls are present, but they do not constitute a published educational-quality benchmark.
Vision and embedding support are optional
Presentation rendering may require LibreOffice, visual extraction requires a reachable compatible model, and dense retrieval requires an Ollama embedding model; failures can reduce coverage while retaining native text.
Model response and batch failures remain possible
Provider availability, reasoning-token limits, malformed JSON and partial visual processing may interrupt requests. The implementation includes retries and failure handling, but it does not guarantee uninterrupted generation.
Source directory.
These source paths are identified in the project implementation or accompanying documentation. Links open the corresponding GitHub source location; availability can change as repositories evolve.
FastAPI endpoints, persistence, streaming and human-controlled actions
backend/app.pyFile validation, document chunks, SQLite FTS5 and optional dense retrieval
backend/documents.pyPDF/Office/image processing with native and optional visual content
backend/ingestion.pyAcademic tool calling, web/course research and approved question planning
backend/agent_mode.pyEditable drafts and selected-question Exam Mode handoff
backend/agent_drafts.pyEvidence selection and inferred complexity/Bloom baselines
backend/exam_context.pyContext-engineered batch generation, split retries and selective review
backend/exam_batch.pyBlueprints, versioned questions, approvals and export gate
backend/exam_builder.pyRead-only Deep Agents enhancement for approved exam slots
backend/exam_planner.py