# CES-Alpha Question JSON File Format — Version 1.0

This document specifies the JSON file that holds one or more auto-graded questions of CES-Alpha. Follow it to write such a file by hand, with a script, or with an AI agent.

- Web page of this document: <https://document.ces-alpha.jp/manual/auto_grade_question_format.html>
- This document as Markdown: <https://document.ces-alpha.jp/manual/auto_grade_question_format.en.md>
- Example files: section 9

## 1. Where the file is used

On the question edit page of a teacher:

- **Download**: the "Question JSON file" zone at the bottom of the page downloads the question as a file of this format ("Download JSON file").
- **Load one file**: "Load question" → "From local json file" fills the form with one question. Nothing is saved until you press the save button.
- **Load several questions**: if you choose several files, a `.zip` file, or a file holding a list of questions, all of them are added as new questions to the group selected on the page.

There are two sites, and a file is for one of them:

| Site | `site` | Questions |
|---|---|---|
| Programming site (`code.ces-alpha.org` and its sub-sites) | `"code"` | Programming questions, and short text answers |
| Mathematics site (`math.ces-alpha.org`) | `"math"` | Math questions graded by symbolic computation |

## 2. File basics

- Encoding: **UTF-8**. Format: JSON ([RFC 8259](https://www.rfc-editor.org/rfc/rfc8259)). Extension: `.json`.
- One question is one JSON **object**. For several questions, use one of:
    - a JSON **array** of question objects in one file,
    - several `.json` files chosen at the same time,
    - a `.zip` file of `.json` files (folders inside are allowed; files starting with `.` and the `__MACOSX` folder are skipped).
- All text values are JSON strings. Write a line break as `\n`. Write a backslash as `\\`, so the LaTeX `\frac{1}{2}` is written `"\\frac{1}{2}"` in the file.
- Keys that this document does not list are ignored.
- An empty string `""` means the same as a missing key.

## 3. Header (required)

Every question object of format 1.0 starts with these two keys:

| Key | Type | Value |
|---|---|---|
| `ces_question_format` | string | `"1.0"`: the format version, as `"major.minor"`. |
| `site` | string | `"code"` or `"math"`: the site the question is for. |

Rules for reading a file:

- **Same major version, any minor version** (`"1.0"`, `"1.2"`, …): the file is read; keys that a later minor version added are ignored.
- **Other major version** (`"2.0"`, …): the file is refused.
- **`site` not the site of the page**: the file is refused. A math question cannot be loaded on the programming site, and the other way round.
- **No `ces_question_format` key**: the file is a file of before 1.0 (see section 8) and is read as before.

## 4. Fields used on both sites

| Key | Type | Required | Default | Meaning |
|---|---|---|---|---|
| `question_type` | string | yes | — | The kind of question. Values: see section 5 (math) and section 6 (code). |
| `question_title` | string | yes\* | `""` | The title shown in question lists. |
| `question_content` | string | yes\* | `""` | The question text shown to students (section 7). |
| `score` | integer | no | `1` | The points of the question ("Score setting"). |
| `evaluation_method` | integer | no | math `0`, code `1` | How the answer is graded. Values: see sections 5 and 6. |
| `question_hint` | string | no | `""` | A hint that students can open ("Hint"). |
| `q_solution_explanation` | string | no | `""` | The explanation of the solution, shown to students with the correct answer ("Solution Explanation"). |
| `note` | string | no | `""` | A note for teachers ("Note"). Students never see it. |

\* At least one of `question_title` and `question_content` must be set. Set both.

## 5. Mathematics site (`"site": "math"`)

### 5.1 Question types

| `question_type` | The student… | Fields used |
|---|---|---|
| `"text"` ("Answer in textbox") | types one or more expressions | `question_answer`, `question_variable`, `answer_const` |
| `"select"` ("Select from list") | chooses one item of a list | `answer_candidate`, `question_answer` |
| `"matrix"` ("Answer as vector or matrix") | types a vector or a matrix | `question_answer`, `question_variable` |

### 5.2 Fields

| Key | Type | Required | Meaning |
|---|---|---|---|
| `question_variable` | string | no | Parameters ("Parameter"): random values that make each student's question different (5.3). |
| `question_answer` | string | yes | The correct answer. Its form depends on `question_type` (5.4). |
| `answer_candidate` | array of strings | `select` only | The choices, in the order they are shown. |
| `answer_const` | boolean | no (`text` only) | `true` ("Allow the addition of a constant"): an answer that differs from `question_answer` by a constant is also correct (indefinite integrals). Default `false`. |

### 5.3 Parameters (`question_variable`)

One parameter per line (`\n` between lines), `name=values`:

```text
a=1,2,5
b_1=-3,...,-1,1,...,5
alpha=1,...,5
beta=a+b_1
```

- A comma-separated list of numbers: one value is drawn at random for each student. `m,...,n` stands for all the integers from `m` to `n`.
- An expression (it has letters, such as `beta=a+b_1`): it is computed from the other parameters.
- Names: one letter (`a`, `b`, `x`, …) or a Greek letter name (`alpha`, `beta`, …; not `lambda`, `pi` or `e`), optionally with a one-digit subscript (`b_1`, `beta_9`).
- `E` (or `e`), `pi` and `I` (the imaginary unit) are the mathematical constants.
- In `question_content`, a parameter inside `$...$` is replaced by its value. Write Greek letters there with a backslash (`$\\alpha$`); in `question_variable` write them without it (`alpha`).

### 5.4 Answer (`question_answer`)

**`text`**: one line for each answer box, `name=expression`:

```text
A=2*a*x+b
B=2*a+b
```

- The name (`A`, `B`) is the label of the answer box. Write it in the question text with `\ans{A}` to show where it goes (section 7).
- The expression is written as on a calculator, with SymPy syntax: `*` for multiplication, `^` or `**` for powers, `sqrt(x)`, `sin(x)`, `exp(x)`, `log(x)`, `pi`, `E`, `I`, `oo` (infinity). Parameters can be used.
- Several correct answers for one box: separate them with `;` (`x=2;-2`).
- A grading method for one line only: add `# mode:1`, `# mode:2` or `# mode:3` at the end of the line (`A=x+1 # mode:3`). The numbers are those of 5.5.

**`matrix`**: one row per line, the elements separated by commas. A column vector is one element per line.

```text
a+2, 2
1, 1
```

**`select`**: the text of the correct choice, exactly as it is written in `answer_candidate`.

### 5.5 Grading method (`evaluation_method`)

| Value | Name in the form | The answer is correct when… |
|---|---|---|
| `0` | [mode:1] Equivalence check | it has the same mathematical meaning: `(x-1)^2` = `x^2-2x+1`. |
| `1` | [mode:2] Formal check | it has the same form, up to order: `(x-3)(x-1)` = `(x-1)(x-3)`, but `3x` ≠ `x+2x`. |
| `2` | [mode:3] Strict check | it is the same string: `2x` ≠ `2*x`. |

Not used for `select`, which always compares the chosen text.

## 6. Programming site (`"site": "code"`)

### 6.1 Question types

| `question_type` | The student… | Graded by |
|---|---|---|
| `"code"` ("Code Creation") | writes a program | running `sample_code` and the student's code with the same input, and comparing the results |
| `"text"` ("Answer in textbox") | types a short answer | comparing it with `question_answer` |

### 6.2 Fields of a `code` question

| Key | Type | Required | Meaning |
|---|---|---|---|
| `code_language` | string | yes | `"Python"` (Python 3), `"R"` or `"MATLAB"` (run with Octave). Default `"Python"`. |
| `answer_template` | string | no | The code that the student's editor starts with. |
| `sample_code` | string | yes | The model answer. It is run on the grading server; students do not see it. |
| `object_for_grade` | string | yes | What is compared (6.3). |
| `input_argument` | string | no | The inputs of a function call, one test case per line (6.3). |
| `evaluation_method` | integer | no | `1` ("Check code-running result"): run the code and compare the results (default). |

### 6.3 What is compared (`object_for_grade`, `input_argument`)

There are two ways.

**A function.** `object_for_grade` is a call, such as `add(a, b)`. Each line of `input_argument` is one test case: its comma-separated values are put in the place of the arguments, in order. The function is called with each test case in `sample_code` and in the student's code, and the return values must match.

```json
"object_for_grade": "add(a, b)",
"input_argument": "1, 2\n-3, 5\n0.5, 0.25"
```

- Lists and nested calls can be values: `[1,2], 5` and `c(1,2,3), 1000` are two values each.
- The placeholders `var1`, `var2`, … (or `var` for a single value) also work, also inside an expression: `solve(var1, np.array([var2, var3]))`.
- Without `input_argument`, the function is called once without arguments (`f()`).

**Variables.** `object_for_grade` is a comma-separated list of variable names, such as `m, mx`. Both programs are run, and the values of these variables at the end must match. `input_argument` is not used.

### 6.4 A `text` question on the programming site

`question_answer` has one line for each answer box, `name=value`, as in 5.4. Several correct answers are separated with `;` (`PrimeNumber=3;13;23`). If the answer itself has a semicolon, write it as `\;` (in the JSON file: `"\\;"`).

## 7. Text of the question, hint and explanation

- `question_content`, `question_hint` and `q_solution_explanation` are **Markdown**. A blank line starts a new paragraph. Code can be written with backticks.
- On the mathematics site, LaTeX formulas can be used: `$...$` inline and `$$...$$` as a block.
- `\ans{name}` draws a box in the text, at the place of the answer `name`.

## 8. Files of before 1.0

Files without `ces_question_format` are read as before. In such files, and only for compatibility, these other names are accepted. Do not use them in new files.

| Key of 1.0 | Other names also accepted |
|---|---|
| `question_type` | `exercise_type`, `q_type` |
| `question_title` | `title` |
| `question_content` | `q_content` |
| `question_variable` | `q_variable` |
| `question_answer` | `answer` |
| `answer_candidate` | `answer_choices` |
| `question_hint` | `hint` |
| `q_solution_explanation` | `solution_explanation` |
| `answer_template` | `code_template` |
| `object_for_grade` | `code_output` |

In a file of before 1.0, the question type is guessed when it is missing or is not a type of the site: on the programming site, a question with `answer_template` or `object_for_grade` is `code`; otherwise it is `text`.

## 9. Examples

| File | Site | Type |
|---|---|---|
| [code_python_function.json](https://document.ces-alpha.jp/manual/question_format_examples/code_python_function.json) | code | `code`, a function with test cases |
| [code_python_variables.json](https://document.ces-alpha.jp/manual/question_format_examples/code_python_variables.json) | code | `code`, variables |
| [code_text.json](https://document.ces-alpha.jp/manual/question_format_examples/code_text.json) | code | `text` |
| [math_text.json](https://document.ces-alpha.jp/manual/question_format_examples/math_text.json) | math | `text` with parameters, two answers |
| [math_text_integral.json](https://document.ces-alpha.jp/manual/question_format_examples/math_text_integral.json) | math | `text` with `answer_const` |
| [math_select.json](https://document.ces-alpha.jp/manual/question_format_examples/math_select.json) | math | `select` |
| [math_matrix.json](https://document.ces-alpha.jp/manual/question_format_examples/math_matrix.json) | math | `matrix` |
| [math_several_questions.json](https://document.ces-alpha.jp/manual/question_format_examples/math_several_questions.json) | math | an array of two questions |

A programming question:

```json
{
  "ces_question_format": "1.0",
  "site": "code",
  "question_type": "code",
  "question_title": "Sum of two numbers",
  "question_content": "Write a function `add(a, b)` that returns the sum of `a` and `b`.",
  "score": 1,
  "evaluation_method": 1,
  "code_language": "Python",
  "answer_template": "def add(a, b):\n    # Write your code here.\n    return None\n",
  "sample_code": "def add(a, b):\n    return a + b\n",
  "object_for_grade": "add(a, b)",
  "input_argument": "1, 2\n-3, 5\n0.5, 0.25",
  "question_hint": "Use the + operator.",
  "q_solution_explanation": "Return `a + b`.",
  "note": "Week 1"
}
```

A math question:

```json
{
  "ces_question_format": "1.0",
  "site": "math",
  "question_type": "text",
  "question_title": "Derivative of a polynomial",
  "question_content": "Let $f(x) = a x^2 + b x$. Find $f'(x)$ and $f'(1)$.\n\n$f'(x) = $ \\ans{A}, $f'(1) = $ \\ans{B}",
  "question_variable": "a=1,...,5\nb=-3,-2,-1,1,2,3",
  "question_answer": "A=2*a*x+b\nB=2*a+b",
  "evaluation_method": 0,
  "score": 2,
  "question_hint": "$(x^n)' = n x^{n-1}$",
  "q_solution_explanation": "$f'(x) = 2ax + b$, so $f'(1) = 2a + b$."
}
```

## 10. Instructions for AI agents

Give this section, or the whole document, to an AI agent that writes question files.

```text
You write question files for CES-Alpha in the "CES-Alpha Question JSON File Format 1.0".
- Output valid JSON only, UTF-8, with no comments and no text around it.
- Each question is an object that starts with "ces_question_format": "1.0" and "site": "code" or "math".
- For several questions, output a JSON array of such objects.
- Use only the keys of the format. Never use the old names of section 8.
- Always set question_type, question_title, question_content and question_answer
  (for a programming "code" question: code_language, sample_code and object_for_grade instead of question_answer).
- Math site: question_type is "text", "select" or "matrix"; expressions use SymPy syntax (2*x, x^2, sqrt(x), pi, E).
  A "text" answer has one line "name=expression" per answer box; mark each box in the text with \ans{name}.
  A "select" answer is exactly one of answer_candidate.
- Programming site: question_type is "code" or "text"; code_language is "Python", "R" or "MATLAB".
  sample_code must run without errors and define what object_for_grade names.
  For a function, give several test cases in input_argument, one per line.
- Escape inside JSON strings: newline as \n, backslash as \\ (LaTeX \frac is written \\frac), double quote as \".
- Check that the answer is correct before writing it.
```

## 11. Changes

| Version | Date | Changes |
|---|---|---|
| 1.0 | 2026-10 | First version: the header `ces_question_format` and `site`; `answer_const`; the old names of section 8 are for compatibility only. |