ENGINEERING NOTEBOOK / ACADEMIC AI AGENTS / 02

QuestGen.

From course evidence to a reviewed exam. ↘

A course-grounded academic workspace with human-controlled assessment generation.

QuestGen is a Next.js 16 and FastAPI application for professors with three connected modes: cited course-material chat, an interactive research-and-question-drafting agent, and an exam-paper builder. A local SQLite workspace stores courses, indexed documents, conversations, plans, question drafts and review history. Source restrictions, approval checkpoints, batch generation and printable PDF/DOCX exports are enforced by backend services rather than entrusted to the language model.

Active development · local, single-user application Source-grounded technical notes
FIG 02 · SYSTEM SKETCH
3connected workspace modesChat · Agent · Exam
5supported question formatsMCQ · short · long · problem-solving · custom
HITLplan, draft & paper approvalsExplicit backend-controlled actions
01
WHY THIS EXISTS

The problem, and the approach.

CHALLENGE

What needed solving?

A useful academic assessment must respect the exact course materials, cover the intended topics, vary cognitive demand across different question formats, and preserve the professor’s ability to change or reject a draft. Sending a whole syllabus to one unconstrained prompt makes source boundaries, marks distribution, question quality and revisions difficult to control.

DIRECTION

What was built?

Separate open-ended academic assistance from question consultation and final paper construction. In Chat Mode, retrieve cited course excerpts. In Agent Mode, discuss requirements, optionally research, propose a versioned plan and wait for explicit approval before drafting editable questions. In Exam Mode, validate the blueprint and selected documents, infer slot-level complexity when unspecified, draft question sets in bounded batches, and let the professor keep, revise, reorder and finally approve the paper for export.

TECHNICAL TOOLKIT
PythonFastAPISQLite / FTS5Next.js 16React 19TypeScriptLangChain Deep Agents (advisory)Tool callingBM25 + optional Ollama embeddingsOptional vision transcriptionPDF / DOCX / PPTX ingestionSSEPDF / DOCX export
02
SYSTEM DESIGN

Architecture, without the noise.

This is a simplified conceptual diagram of the components documented in the repository; it is not a claim that all operations use a single synchronous pipeline.

FIG. 02 / SYSTEM FLOW
01 / STAGECourse library

Upload PDF, PPTX, DOCX, text and image files; native extraction plus optional model-assisted visual reading.

02 / STAGEIndexed local context

SQLite stores courses, threads, chunks and versions; FTS5/BM25 search with optional Ollama dense-embedding fusion.

03 / STAGEChat / Agent / Exam

Cited Q&A; tool-driven consultation and editable drafts; approved blueprint and source-scoped paper building.

04 / STAGEPlanning & generation

Agent plan approvals or Exam Mode slot planning; context-engineered, mixed-format batch generation and validation.

05 / STAGEProfessor review

Select, edit, regenerate, accept, reorder or reject questions; keep versions and require final approval.

06 / STAGEPaper export

FastAPI exports student or instructor versions as PDF or DOCX only after paper approval.

Source: https://github.com/cRED-f/QuestGen-AI-Agent · Architecture summarized from repository implementation and documentation.
03
RUNBOOK

How it works, step by step.

  1. 01

    Ingest and index the course

    Create a course, upload supported material and extract ordered content. PDF pages and presentation slides can receive optional visual transcription. Persist chunks and searchable metadata in SQLite; report coverage and warnings when visual analysis is incomplete.

    backend/ingestion.py
  2. 02

    Retrieve under the chosen source scope

    Use course-scoped SQLite FTS5/BM25 search, optionally fused with local Ollama embeddings. When files are explicitly selected, pass the allowed document IDs into retrieval and exam context preparation instead of searching the whole library.

    backend/documents.py
  3. 03

    Consult in Agent Mode

    Discuss educational goals and question mix rather than applying a fixed MCQ checklist. A streaming tool-calling agent can inspect course material and perform bounded web research, then save a versioned question plan that requires an explicit UI approval.

    backend/agent_mode.py
  4. 04

    Draft and revise questions

    After approved consultation, store complete question drafts with answers and explanations. The professor can edit, revise, select and carry chosen questions into Exam Mode without starting a second agent drafting pipeline.

    backend/agent_drafts.py
  5. 05

    Plan and draft a full paper in Exam Mode

    Validate section counts, marks and source scope. Build slot-specific evidence and infer difficulty/Bloom demand when unspecified; optional Deep Agents planning remains advisory. Generate mixed-format questions in batches, splitting invalid or oversized batches for retries.

    backend/exam_batch.py
  6. 06

    Review, finalize and export

    Keep or revise selected drafts, maintain question versions and ordering, then explicitly approve the complete paper. Student and instructor exports are available as PDF or DOCX.

    backend/exam_builder.py
04
ENGINEERING THINKING

The choices behind the system.

NOTE 01

Three different jobs, not five legacy agents

Chat Mode answers and cites materials, Agent Mode handles consultation/research/drafts, and Exam Mode handles validated assessment structure and printing. The old Extractor → Creator → Analysis → Decider → Formatter chain is not this codebase’s architecture.

NOTE 02

Human approval belongs to FastAPI

A model can suggest a plan, topic coverage, question content or a revision. Only explicit requests from the professor can approve a consultation plan, accept paper drafts, or authorize final export.

NOTE 03

One context-engineered batch, then selective revisions

Exam Mode groups mixed question formats into larger generation requests, checks slot IDs and source citations, splits malformed batches when necessary, and preserves already accepted questions instead of invoking full reasoning separately for each item.

NOTE 04

Complexity is inferred, not demanded

When a professor leaves difficulty on auto, the planning baseline considers marks, question type and assessment purpose. An optional Deep Agents planner may enrich topics and Bloom levels but cannot override explicitly set difficulty or change the approved blueprint.

05
TECHNICAL DETAILS

Inside the implementation.

Source grounding, consultation and approval

01

Hybrid retrieval with a lexical fallback

SQLite FTS5/BM25 runs locally. If QUESTGEN_EMBED_MODEL is configured, Ollama embeddings add a dense candidate set with reciprocal-rank fusion; lexical retrieval remains available without embeddings.

02

Multimodal lecture materials

PDF, DOCX, PPTX, TXT, Markdown and common image formats are accepted. Native extraction is preserved if optional vision transcription or slide rendering cannot verify diagrams and scanned content.

03

Consultation and research boundaries

Agent Mode’s tool-calling loop supports course search, up to two web searches and three page reads per turn, versioned plan proposals, approved draft creation and individual question revisions.

04

Exam Mode invariants

The professor approves exact marks/format sections and selects all-course, selected-file or general-knowledge scope. FastAPI validates slot counts, uniqueness, citations and versions, with a final paper signature for export.

05

Configurable model access

Backend provider connections include OpenAI, Anthropic, OpenRouter, 9router and Ollama; stored credentials are encrypted locally. This is a configurable model API, not a claimed fully offline product.

BACKEND SOURCEPython · source excerpt

Enforcing source-scoped exam evidence

Excerpt from the uploaded implementation. Exam Mode retrieves against the approved document IDs and rejects evidence outside that set before question drafting.

found = await asyncio.to_thread(
    documents.retrieve, session['course_id'], section['topic'], 12, material_ids)
if not found:
    found = await asyncio.to_thread(
        documents.sample_course_sources, session['course_id'], 12, material_ids)
if not found:
    raise ValueError(f'No indexed evidence for section {section["id"]}. Reprocess the course files.')
if any(s.get('document_id') and s['document_id'] not in material_ids for s in found):
    raise ValueError('Retrieval returned an unapproved document')
SOURCE REFERENCE / backend/exam_context.py · build()
06
MEASURED EVIDENCE

Results in context.

Implementation evidence, not an accuracy benchmark

The uploaded code contains FastAPI and frontend tests for consultation approval, Agent-to-Exam handoff, source restrictions, ingestion resilience, asynchronous batch generation, question editing/versioning, and paper export. These automated tests check behavior and constraints, not independent learning outcomes or model-generated question correctness. The source archive does not establish a measured hallucination rate, teacher time saved, production uptime or human-rated assessment quality; no such metric is claimed here.

Numbers here are reported by the project author in the linked public repository or CV. They are not independent tests performed for this portfolio.

07
ENGINEERING HONESTY

Trade-offs & open questions.

01

Single-user local security model

The FastAPI service explicitly does not implement multi-user authentication. Course separation is logical; it must not be exposed publicly as a multi-tenant service without access controls.

02

Question accuracy still requires professor review

Model-produced answers, complexity assignments and citations can be wrong or pedagogically weak. Review controls are present, but they do not constitute a published educational-quality benchmark.

03

Vision and embedding support are optional

Presentation rendering may require LibreOffice, visual extraction requires a reachable compatible model, and dense retrieval requires an Ollama embedding model; failures can reduce coverage while retaining native text.

04

Model response and batch failures remain possible

Provider availability, reasoning-token limits, malformed JSON and partial visual processing may interrupt requests. The implementation includes retries and failure handling, but it does not guarantee uninterrupted generation.

08
READ THE CODE

Source directory.

These source paths are identified in the project implementation or accompanying documentation. Links open the corresponding GitHub source location; availability can change as repositories evolve.