Perturbation-Driven Visual Analytics for Probing InternalRepresentations in Neural Language Models

• Eeshwar Pasula

  University of Texas at Arlington

  Author

This paper introduces an interactive visual analytics framework designed to support exploratory analysis of neural network models for natural language inference through a perturbation-driven paradigm. Rather than treating trained models as static black boxes, the system enables users to dynamically manipulate inputs, internal attention mechanisms, and output predictions while observing corresponding changes across the processing pipeline. The interface integrates multiple coordinated views including bipartite graph and matrix representations of attention, barycentric coordinate plots for probabilistic predictions, and a pipeline visualization for tracking parameter updates to support hypothesis formation and causal reasoning about model behavior. A constrained optimization procedure, inspired by the margin-infused relaxed algorithm, allows users to correct erroneous predictions while minimizing parameter deviation and to compare the relative influence of encoder, attention, and classifier stages on prediction outcomes. The system also overlays syntactic dependency structures onto attention visualizations, enabling grammar-guided simplification and facilitating investigation of the relationship between linguistic structure and learned alignments. Evaluation with NLP researchers demonstrates that the tool supports a range of analytical tasks, including stability assessment, error diagnosis, attention editing, and comparative analysis of model components. The framework is implemented as a lightweight Python library that integrates with existing PyTorch models, lowering the barrier to adoption for routine model interrogation.

• PDF

Copyright (c) 2026 International Journal of Intelligent Systems and Data Science

This work is licensed under a Creative Commons Attribution 4.0 International License.