In a randomized experiment with more than 1,000 students, access to ChatGPT and training in causal reasoning improved student work in complementary ways.

What happens when students use ChatGPT on a real-world assignment? Do quality improvements come at the expense of originality?

A new experiment from researchers at Bocconi University, in collaboration with OpenAI Economic Research, found distinct and complementary effects from ChatGPT access and critical-thinking training. Access to ChatGPT improved the quality and coherence of students’ work, while an exercise in causal reasoning—a form of critical thinking—led students to generate more unique ideas. Students who received both ChatGPT access and the training showed both effects.

The results also highlight the importance of a holistic approach in assessing student progress. As AI makes it easier for students to produce polished answers, assignments may need to adapt to measure other desired qualities such as originality.

During the experiment, more than 1,000 first-year undergraduate students at Bocconi University worked on a real-world business case developing marketing recommendations for the university’s merchandise store.

Students were randomly assigned by class period into one of four groups that received: access to ChatGPT (GPT‑4o), training in causal reasoning, both, or neither. Causal reasoning is a specific form of critical thinking related to linking cause and effect and explaining why a given solution may or may not work. The training students received was unrelated to AI, instead teaching students causal reasoning concepts through an exercise that involved a game, examples, questions, and feedback.

Student submissions were evaluated by trained human graders using a five-point rubric.

Source link