Fix/prompt updates

[m-messer]

Problem

The evaluation function made a single combined LLM call that packed moderation, correctness, and feedback into one prompt and one JSON schema. This meant:

• A clearly manipulative/off-topic submission still paid for a full correctness + feedback generation pass before being rejected.
• Moderation, correctness, and feedback instructions were mashed into one prompt, making each concern harder to reason about and tune independently.
• One JSON parse failure anywhere in the combined response threw away the entire evaluation, with no way to tell which part failed.
• model, main_prompt, default_prompt, and feedback_prompt were all required params — omitting any of them threw an unhandled KeyError instead of falling back to something
  sensible.
• The {{question}}/main_prompt/default_prompt/feedback_prompt/moderator_prompt param names were inconsistent and didn't clearly convey what each one did.
• Teacher-facing docs (docs/user.md) were still the unfilled template.

Solution

Split the single call into three sequential, single-purpose LLM calls (moderation → correctness → feedback), short-circuiting as soon as one fails, and gave every non-essential
param a reasonable default so a minimal {model, context}-less request still works. Renamed params for clarity and brought docs/tests in line with the new behaviour.

Changes

• evaluation_function/evaluation.py
  • Split the combined call into check_moderation → check_correctness → generate_feedback, each with its own JSON schema (passes_moderation, is_correct, feedback) and its own
    parse-failure handling (_request_json, _failure_result).
  • Moderation now short-circuits immediately on failure, skipping the correctness/feedback calls entirely.
  • Added defaults so model, correctness_decision, and feedback_guidance are all optional: DEFAULT_MODEL (openai/gpt-4o-mini), DEFAULT_FEEDBACK_GUIDANCE, and a context-aware
    default_correctness_decision() that picks between DEFAULT_CORRECTNESS_DECISION_WITH_CONTEXT / ..._NO_CONTEXT depending on whether context was supplied (avoids a dangling "to the
    following question:" when it's missing).
  • Renamed params: question → context, main_prompt → correctness_decision, feedback_prompt → feedback_guidance, moderator_prompt → moderation_prompt; dropped default_prompt
    (folded into the fixed JSON-output instructions each call appends).
  • Wrapped the evaluation body in try/finally so the OpenAI client is always closed, including on early-return/exception paths.
• Dockerfile — raised FUNCTION_WORKER_SEND_TIMEOUT from 90s to 120s to give the three sequential calls enough headroom.
• evaluation_function/dev.py — updated the local CLI's DEFAULT_PARAMS to the renamed param names.
• evaluation_function/evaluation_test.py — rewrote tests for the three-call flow (per-step mocking via _patch_openai), and added coverage for: default model, default
  correctness/feedback prompts when omitted, the context-free correctness default, moderation short-circuiting, and per-step parse-failure handling.
• README.md — updated "How It Works" for the three-call flow, the request-structure table for the renamed/optional params, and all example requests.
• docs/user.md — replaced the placeholder with a teacher-facing summary of the three-call flow and how {{answer}}/{{context}} are populated.

[peterbjohnson]

Looks fine. I've commented on some minor things.