README.md

Progress Measures Simulations

This repository contains the simulation code accompanying the paper "Measuring Skillful Effort: Progress Bar for Practice in Learning Environments".

It provides a simulation framework for evaluating progress measures, with the code structure closely aligned to the sections of the paper:

• models.py

  • Defines models of students and items, using a logistic function to determine the probability of a correct answer.
  • Corresponds to Section 5.1.

• progress_measures.py

  • Implements the individual progress measures analyzed in the paper.
  • Corresponds to Section 4.
  • The code uses the variable status to represent progress, which corresponds to the mathematical notation $p_t$ in the paper.

• evaluation_metrics.py

  • Defines the evaluation metrics used to compare and optimize progress measures.
  • Corresponds to the equations shown in Figure 9.

• config.py

  • Specifies simulation scenarios corresponding to Table 2.
  • Defines threshold values used in evaluation metrics (visualized in the left part of Figure 9).
  • Includes parameter settings for the progress measures found via the optimization procedure (reported in Table 3).

• optimization.py

  • Implements the optimization of progress measure parameters using the optuna library.

• simulation.py

  • Contains the code used to generate the analysis results presented in the paper (Figure 4, Figure 8, Figure 10, Figure 11, Table 4).