PhD-written STEM reasoning problems dataset
Hard problems for capable models: graduate and research-level questions in physics, mathematics, chemistry, biology, computer science and engineering, each written by a PhD in the field and solved again, independently, by another. Every problem comes with one final answer a program can check and a full worked solution.
Specification
- Fields
- Physics, mathematics, chemistry, biology, computer science and engineering, at graduate and research level
- Each problem
- Statement, one final answer, a full worked solution, field and sub-field, and a difficulty rating
- Answers
- Checkable by a program: exact integers, numbers with a tolerance and units, symbolic expressions tested for equivalence, or code run against tests
- Writers
- PhDs whose credentials are checked, each writing problems in their own field
- Verification
- A second PhD solves every problem without seeing the answer; only problems where both agree are kept
- Format
- JSONL, with LaTeX for mathematics and images for figures where a problem needs one
Example of the data
What you see, and the file you get.
| Quantum mechanics | ΔE = V₀ / 2 | symbolic · 2nd PhD agrees |
|---|---|---|
| Statistical mechanics | k_B T_c / J = 2.26919 | numeric ± 1e-4 |
| Algebra | 116 polynomials | exact integer |
| problem_id | field | problem | final_answer | answer_type | second_solver |
|---|---|---|---|---|---|
| math_0118 | mathematics · algebra | How many monic irreducible polynomials of degree 6 are there over the field with 3 elements? | 116 | integer | agrees |
| phys_0042 | physics · statistical mechanics | Give k_B T_c / J for the ferromagnetic Ising model on the square lattice in zero field. | 2.26919 | number ±1e-4 | agrees |
| phys_0077 | physics · quantum mechanics | A particle in an infinite well of width L feels an extra potential V0 on the left half only. What is the first-order shift of the ground-state energy? | V0/2 | expression | agrees |
| prob_0019 | mathematics · probability | Expected number of steps for a simple random walk on a 12-vertex cycle to visit every vertex. | 66 | integer | agrees |
| chem_0031 | chemistry · stereochemistry | How many stereoisomers does 2,3,4-trihydroxyglutaric acid have? | 4 | integer | agrees |
| cs_0054 | computer science · algorithms | Fewest comparisons that sort any 5 distinct keys in the worst case. | 7 | integer | agrees |
Sample rows. Also as JSON or Excel.
Contact us to get the dataset
Tell us what you need. We walk you through the data, its formats and how it is checked on a short call.
What it is used for
Reasoning benchmarks
Test models on problems that need real expertise, scored automatically.
Reinforcement learning with verifiable rewards
Reward a model only when its final answer checks out.
Fine-tuning on worked solutions
Train on solutions written step by step by people who do this work.
Progress by field
See where a model is strong or weak, field by field.
Questions
Who writes the problems?
PhDs whose credentials are checked, each in their own field. A second PhD solves every problem without seeing the answer, and only problems where both agree are kept.
How are answers checked automatically?
Each problem has one final answer in a checkable form: an exact integer, a number with a tolerance, a symbolic expression tested for equivalence, or code run against tests.
Can I see samples first?
Yes. Request samples with the form on this page and we will send them with the full specification.
Tell us what your model needs.
Request samples and the full specification. We reply within a couple of business days.
Request samples