Visual Analogue scales (VASs) are increasingly popular in psychological, social, and medical research. However, VASs can also be more demanding for respondents, potentially leading to quicker disengagement and a higher risk of careless responding. Existing mixture modeling approaches for careless response detection have so far only been available for Likert-type and unbounded continuous data but have not been tailored to VAS data. This study introduces and evaluates a model-based approach specifically designed to detect and account for careless respondents in VAS data. We integrate existing measurement models for VASs with mixture item response theory models for identifying and modeling careless responding. Simulation results show that the proposed model effectively detects careless responding and recovers key parameters. We illustrate the model’s potential for identifying and accounting for careless responding using real data from both VASs and Likert scales. First, we show how the model can be used to compare careless responding across different scale types, revealing a higher proportion of careless respondents in VAS compared to Likert scale data. Second, we demonstrate that item parameters from the proposed model exhibit improved psychometric properties compared to those from a model that ignores careless responding. These findings underscore the model’s potential to enhance data quality by identifying and addressing careless responding.

In maximum likelihood factor analysis, we need to solve a complicated system of algebraic equations, known as the normal equation, to get maximum likelihood estimates (MLEs). Since this equation is difficult to solve analytically, its solutions are typically computed with continuous optimization methods, such as the Newton–Raphson method. With this procedure, however, the MLEs are dependent on initial values since the log-likelihood function is highly non-concave. Particularly, the estimates of unique variances can result in zero or negative, referred to as improper solutions; in this case, the MLE can be severely unstable. To delve into the issue of the instability, we algebraically compute all candidates for the MLE. We provide an algorithm based on algebraic computations that is carefully designed for maximum likelihood factor analysis. To be specific, Gröbner bases are employed, powerful tools to get simplified sub-problems for given systems of algebraic equations. Our algebraic algorithm provides the MLE independent of the initial values. While computationally demanding, our algebraic approach is applicable to small-scale problems and provides valuable insights into the characterization of improper solutions. For larger-scale problems, we provide numerical methods as practical alternatives to the algebraic approach. We perform numerical experiments to investigate the characteristics of the MLE with our two approaches.

Robert Joseph Mislevy (known to friends and colleagues as Bob) was born June 28, 1950, and passed away at the age of 74 in Severna Park, Maryland, USA on May 22, 2025, with his daughters by his side. In his career, Bob had tremendous impact on research, practice, and philosophy. He was a past-president of the Psychometric Society, an elected member of the National Academy of Education, an elected fellow of the American Educational Research Association, and served on National Academies of Science committees concerning assessment, instruction, and psychology. Among other honors, he received the Award for Significant Contribution to Educational Measurement and Research Methodology from the American Educational Research Association, the E.F. Lindquist Award from the American Educational Research Association and ACT, the National Council on Measurement in Education’s Award for Exceptional Achievement in Educational Measurement (five times), and awards for career and lifetime contributions from the Psychometric Society and the National Council on Measurement in Education.

Empathic accuracy (EA) is the ability to accurately understand another person’s thoughts and feelings, which is crucial for social and psychological interactions. Traditionally, EA is assessed by comparing a perceiver’s moment-to-moment ratings of a target’s emotional state with the target’s own self-reported ratings at corresponding time points. However, misalignments between these two sequences are common due to the complexity of emotional interpretation and individual differences in behavioral responses. Conventional methods often ignore or oversimplify these misalignments, for instance by assuming a fixed time lag, which can introduce bias into EA estimates. To address this, we propose a novel alignment approach that captures a wide range of misalignment patterns. Our method leverages the square-root velocity framework to decompose emotional rating trajectories into amplitude and phase components. To ensure realistic alignment, we introduce a regularization constraint that limits temporal shifts to ranges consistent with human perceptual capabilities. This alignment is efficiently implemented using a constrained dynamic programming algorithm. We validate our method through simulations and real-world applications involving video and music datasets, demonstrating its superior performance over traditional techniques.

The article proposes a novel item response theory model to handle continuous responses and sparse polytomous responses in psychological and educational measurement. The model extends the traditional two-parameter logistic model by incorporating a precision parameter, which, along with a beta distribution, forms an error component that accounts for the response continuity. Furthermore, transforming ordinal responses to a continuous scale enables the fitting of polytomous item responses while consistently applying three parameters per item for model parsimony. The model’s accuracy, stability, and computational efficiency in parameter estimation were examined. An empirical application demonstrated the model’s effectiveness in representing the characteristics of continuous item responses. Additionally, the model’s applicability to sparse polytomous data was supported by cross-validation results from another empirical dataset, which indicates that the model’s parsimony can enhance model-data fit compared to existing polytomous models.

With increasingly available computer-based or online assessments, researchers have shown keen interest in analyzing log data to improve our understanding of test takers’ problem-solving processes. In this article, we propose a multi-state survival model (MSM) to action sequence data from log files, focusing on modeling test takers’ reaction times between actions, in order to investigate which factors and how they influence test takers’ transition speed between actions. We specifically identify the key actions that differentiate correct and incorrect answers, compare transition probabilities between these groups, and analyze their distinct problem-solving patterns. Through simulation studies and sensitivity analyses, we evaluate the robustness of our proposed model. We demonstrate the proposed approach using problem-solving items from the Programme for the International Assessment of Adult Competencies (PIAAC).

The latent Markov model (LMM) has been increasingly used to analyze log data from computer-interactive assessments. An important consideration in applying the LMM to assessment data is measurement effects of items. In educational and psychological assessment, items exhibit distinct psychometric qualities and induce systematic variance to assessment outcome data. The current development in LMM, however, assumes that items have uniform effects and do not contribute to the variance of measurement outcomes. In this study, we propose a refinement of LMM that relaxes the measurement invariance constraint and examine empirical performance of the new framework through numerical experimentation. We modify the LMM for noninvariant measurements and refine the inferential scheme to accommodate the event-specific measurement effects. Numerical experiments are conducted to validate the proposed inference methods and evaluate the performance of the new framework. Results suggest that the proposed inferential scheme performs adequately well in retrieving the model parameters and state profiles. The new LMM framework demonstrated reliable and stable performance in modeling latent processes while appropriately accounting for items’ measurement effects. Compared with the traditional scheme, the refined framework demonstrated greater relevance to real assessment data and yielded more robust inference results when the model was ill-specified. The findings from the empirical evaluations suggest that the new framework has potential for serving large-scale assessment data that exhibit distinct measurement effects.

Rating procedure is crucial in many applied fields (e.g., educational, clinical, emergency). In these contexts, a rater (e.g., teacher, doctor) scores a subject (e.g., student, doctor) on a rating scale. Given raters’ variability, several statistical methods have been proposed for assessing and improving the quality of ratings. The analysis and the estimate of the Intraclass Correlation Coefficient (ICC) are major concerns in such cases. As evidenced by the literature, ICC might differ across different subgroups of raters and might be affected by contextual factors and subject heterogeneity. Model estimation in the presence of heterogeneity has been one of the recent challenges in this research line. Consequently, several methods have been proposed to address this issue under a parametric multilevel modelling framework, in which strong distributional assumptions are made. We propose a more flexible model under the Bayesian nonparametric (BNP) framework, in which most of those assumptions are relaxed. By eliciting hierarchical discrete nonparametric priors, the model accommodates clusters among raters and subjects, naturally accounts for heterogeneity, and improves estimates’ accuracy. We propose a general BNP heteroscedastic framework to analyze continuous and coarse rating data and possible latent differences among subjects and raters. The estimated densities are used to make inferences about the rating process and the quality of the ratings. By exploiting a stick-breaking representation of the discrete nonparametric priors, a general class of ICC indices might be derived for these models. Our method allows us to independently identify latent similarities between subjects and raters and can be applied in precise education to improve personalized teaching programs or interventions. Theoretical results about the ICC are provided together with computational strategies. Simulations and a real-world application are presented, and possible future directions are discussed.

We introduce a novel regularization method for detecting differential item functioning (DIF) in two-parameter logistic (2PL) models. Existing regularization methods require choosing a reference group and using an $L_1$ penalty (LP) to shrink the item parameters of focal groups toward those of the reference. This approach has two key limitations: (1) shrinking all focal groups toward a reference is inherently unfair, as results are affected by the choice of reference and direct comparison among focal groups is unavailable and (2) the LP leads to biased estimates because it overly shrinks large nonzero parameters toward zero. These limitations are particularly problematic for intersectional DIF, where various identity aspects intersect to create multiple smaller groups. Our method addresses these issues by penalizing item parameter differences between all pairs of groups using a truncated LP, thereby treating groups equally and avoiding excessive penalization of large differences. Simulations demonstrate that the proposed method outperforms existing approaches by accurately identifying items exhibiting DIF even with multiple small groups. Application to two real-world datasets further illustrates its utility. We recommend this method as a more equitable and precise tool for DIF detection. The proposed method is available as D2PL_pair_em() in the R package VEMIRT (https://map-lab-uw.github.io/VEMIRT).

Process data, in particular, log data collected from a computerized test, documents the sequence of actions performed by an examinee in pursuit of solving a problem, affording an opportunity to understand test-taking behavioral patterns that account for demographic group differences in key outcomes of interest, for instance, final score on a cognitive item. Addressing this aim, this article proposes a latent class mediation analysis procedure. Using continuous process features extracted from action sequence data as indicators, latent classes underlying the test-taking behavior are identified in a latent class mediation model, where an examinee’s nominal latent class membership enters as the mediator between the observed grouping and outcome variables. A headlong search algorithm for selecting the subset of process features that maximizes the total indirect effect of the latent class mediator is implemented. The proposed procedure is validated with a series of simulations. An application to a large-scale assessment highlights how the proposed method can be used to explain performance gaps between students with learning disability and their typically developing peers on the National Assessment of Educational Progress (NAEP) math assessment.

The envelope model has gained significant attention since its proposal, offering a fresh perspective on dimension reduction in multivariate regression models and improving estimation efficiency. One of its appealing features is its adaptability to diverse regression contexts. This article introduces the integration of envelope methods into the factor analysis model. In contrast to previous research primarily focused on the frequentist approach, the study proposes a Bayesian approach for estimation and envelope dimension selection. A Metropolis-within-Gibbs sampling algorithm is developed to draw posterior samples for Bayesian inference. A simulation study is conducted to illustrate the effectiveness of the proposed method. Additionally, the proposed methodology is applied to the ADNI dataset to explore the relationship between cognitive decline and the changes occurring in various brain regions. This empirical application further highlights the practical utility of the proposed model in real-world scenarios.

Diagnostic classification models assume the existence of latent attribute profiles, the possession of which increases the probability of responding correctly to questions requiring the corresponding attributes. Through the use of longitudinally administered exams, the degree to which students are acquiring core attributes over time can be assessed. While past approaches to longitudinal diagnostic classification modeling perform inference on the overall probability of acquiring particular attributes, there is particular interest in the relationship between student progression and student covariates such as intervention effects. To address this need, we propose an integrated Bayesian model for student progression in a longitudinal diagnostic classification modeling framework. Using Pòlya-gamma augmentation with two logistic link functions, we achieve computationally efficient posterior estimation with a conditionally Gibbs sampling procedure. We show that this approach achieves accurate parameter recovery when evaluated using simulated data. We also demonstrate the method on a real-world educational testing data set.

Many observational studies often involve multiple levels of treatment assignment. In particular, fuzzy regression discontinuity (RD) designs have sequential treatment assignment processes: first based on eligibility criteria, and second, on (non-)compliance rules. In such fuzzy RD designs, researchers typically use either an intent-to-treat approach or an instrumental variable-type approach, and each is subject to both overlapping and unique biases. This article proposes a new evidence factors (EFs) framework for fuzzy RD designs with sequential treatment assignments, which may be influenced by different levels of decision-makers. Each of the proposed EFs aims to test the same causal null hypothesis while potentially being subject to different types of biases. Our proposed framework utilizes the local RD randomization and randomization-based inference. We evaluate the effectiveness of our proposed framework through simulation studies and two real datasets on pre-kindergarten programs and testing accommodations.

Teamwork relies on collaboration to achieve goals that exceed individual capabilities, with team cognition playing a key role by integrating individual expertise and shared understanding. Identifying the causes of inefficiencies or poor team performance is critical for implementing targeted interventions and fostering the development of team cognition. This study proposes a teamwork cognitive diagnostic modeling framework comprising 12 specific models—collectively referred to as Team-CDMs—which are designed to capture the interdependence among team members through emergent team cognitions by jointly modeling individual cognitive attributes and a team-level construct, termed teamwork quality, which reflects the social dimension of collaboration. The models can be used to identify strengths and weaknesses in team cognition and determine whether poor performance arises from cognitive deficiencies or social issues. Two simulation studies were conducted to assess the psychometric properties of the models under diverse conditions, followed by a teamwork reasoning task to demonstrate their application. The results showed that Team-CDMs achieve robust parameter estimation, effectively diagnose individual attributes, and assess teamwork quality while pinpointing the causes of poor performance. These findings underscore the utility of Team-CDMs in understanding, diagnosing, and improving team cognition, offering a foundation for future research and practical applications in teamwork-based assessments.

Assessing fit in common factor models solely through the lens of mean and covariance structures, as is commonly done with conventional goodness-of-fit (GOF) assessments, may overlook critical aspects of misfit, potentially leading to misleading conclusions. To achieve more flexible fit assessment, we extend the theory of generalized residuals (Haberman & Sinharay, 2013), originally developed for models with categorical data, to encompass more general measurement models. Within this extended framework, we propose several fit test statistics designed to evaluate various parametric assumptions involved in common factor models. The examples include assessing the distributional assumptions of latent variables and the functional form assumptions of individual manifest variables. The performance of the proposed statistics is examined through simulation studies and an empirical data analysis. Our findings suggest that generalized residuals are promising tools for detecting misfit in measurement models, often masked when assessed by conventional GOF testing methods.

Hidden Markov models (HMMs) are popular for modeling complex, longitudinal data. Existing identifiability theory for conventional HMMs assume emission probabilities are constant over time and the Markov chain governing transitions among the hidden states is irreducible, which are assumptions that may not be applicable in all educational and psychological research settings. We generalize existing conditions on homogeneous HMMs by considering heterogeneous HMMs with time-varying emission probabilities and the potential for absorbing states. Researchers are investigating a family of models known as restricted HMMs (RHMMs), which combine HMMs and restricted latent class models (RLCMs) to provide fine-grained classification of educationally and psychologically relevant attribute profiles over time. These RHMMs leverage the benefits of RLCMs and HMMs to understand changes in attribute profiles within longitudinal designs. The identifiability of RHMM parameters is a critical issue for ensuring successful applications and accurate statistical inference regarding factors that impact outcomes in intervention studies. We establish identifiability conditions for RHMMs. The new identifiability conditions for heterogeneous HMMs and RHMMs provide researchers insights for designing interventions. We discuss different types of assessment designs and the implications for practice. We present an application of a heterogeneous HMM to daily measures of positive and negative affect.

Testitems using open-ended response formats can increase an instrument’s construct validity. However, traditionally, their application in educational testing requires human coders to score the responses. Manual scoring not only increases operational costs but also prohibits the use of evidence from open-ended items to inform routing decisions in adaptive designs. Using machine learning and natural language processing, automatic scoring provides classifiers that can instantly assign scores to text responses. Although optimized for agreement with manual scores, automatic scoring is not perfectly accurate and introduces an additional source of error into the response process, leading to a misspecification of the measurement model used with the manual score. We propose two joint models for manual and automatic scores of automatically scored open-ended items. Our models extend a given model from Item Response Theory for the manual scores by a component for the automatic scores, accounting for classification errors. The models were evaluated using data from the Programme for International Student Assessment (2012) and simulated data, demonstrating their capacity to mitigate the impact of classification errors on ability estimation compared to a baseline that disregards classification errors.

Neuroimaging studies, such as the Human Connectome Project (HCP), often collect multifaceted data to study the human brain. However, these data are often analyzed in a pairwise fashion, which can hinder our understanding of how different brain-related measures interact. In this study, we analyze the multi-block HCP data using data integration via analysis of subspaces (DIVAS). We integrate structural and functional brain connectivity, substance use, cognition, and genetics in an exhaustive five-block analysis. This gives rise to the important finding that genetics is the single data modality most predictive of brain connectivity, outside of brain connectivity itself. Nearly 14% of the variation in functional connectivity (FC) and roughly 12% of the variation in structural connectivity (SC) is attributed to shared spaces with genetics. Moreover, investigations of shared space loadings provide interpretable associations between particular brain regions and drivers of variability. Novel Jackstraw hypothesis tests are developed for the DIVAS framework to establish statistically significant loadings. For example, in the (FC, SC, and substance use) subspace, these novel hypothesis tests highlight largely negative functional and structural connections suggesting the brain’s role in physiological responses to increased substance use. Our findings are validated on genetically relevant subjects not studied in the main analysis.