Mechanistic Interpretability for VLA models
What can we learn about vision language action models from its weights?
My collaborators and I recreated Mechanistic Interpretability for Vision Language Action Models by Bear Häon, Kaylene Stocking et al. to make sense of the hidden circuitry in VLA models. We’re projecting the weight vectors of the VLA’s forward layers into its token space to see what it’s thinking at each layer, and injecting different weight vectors to see how the robot behaves differently.
The project was done as part of the final course project in 11-785 Introduction to Deep Learning that I took my sophomore year of undergrad. Admittedly, much more could be done to make the experiments more rigorous if we weren’t under such a tight deadline.