AI and Experimental Methods*

*New course running in Methods School 2025!

This five-day intensive course provides social science researchers with a comprehensive introduction to the use of Artificial Intelligence (AI) in experimental research. Through hands-on exercises, ethical discussions, and real-world applications, participants will explore how AI can aid in the design, implementation, and analysis of social science experiments. Topics include using AI to generate experimental stimuli, automate coding and analysis, enhance randomization techniques, and improve survey and conjoint experiments. The course emphasizes practical takeaways, equipping participants with actionable skills to integrate AI into experimental research while ensuring methodological rigor, transparency, and ethical responsibility. By the end of the course, participants will be prepared to critically and effectively incorporate AI into their experimental workflows.

 

Dates

This one-week, 17.5-hour course runs Monday-Friday, 7 - 11 July, 2025. The course is scheduled for 1:30 pm - 5:00 pm.

 

Classroom Location

Faculty of Arts and Social Sciences

 

Instructor

Charles Crabtree, Dartmouth College

 

Detailed Description

This course focuses on the integration of AI tools—especially LLMs—into experimental design, implementation, and analysis in the social sciences. Participants will gain hands-on experience using LLMs to generate survey questions, develop and pre-test treatments, code open-ended responses, and conduct exploratory text analysis. The course places special emphasis on validity, transparency, and replicability when using AI-assisted methods in experimental research.

By the end of the course, participants will:

  • Design and refine experiments using AI tools.
  • Use LLMs to develop realistic, valid experimental stimuli.
  • Use LLMs to conduct structured and semi-structured interviews in experimental contexts.
  • Apply AI to analyze qualitative and textual data from experiments.
  • Understand and address ethical concerns related to AI in experimentation.

 

Prerequisites

Participants should have some brief familiarity with experimental design in the social sciences (e.g., conjoint, vignette, survey, or lab experiments), and with academic writing and reading peer-reviewed articles. No prior programming experience or AI expertise required. Knowledge of R and Qualtrics is helpful but not required.

 

Requirements

Participants are:

  • Required to attend all lectures and hands-on lab sessions.
  • Expected to have access to an internet-connected computer and to be able to use the statistical software R for this course.
  • Required to complete short exercises on AI interviewing, treatment generation, outcome coding, and AI-assisted analysis. (During the course)
  • Required to submit a mini-proposal for an AI-assisted experimental design. (During the course)
  • Required to participate in small-group critiques and class discussions. (During the course)

 

Readings

Readings are to be determined given ongoing developments in AI, but will feature a mix of methodological introductions and substantive applications.