Jwala Dhamala

Senior Applied Scientist, AGI Foundations, Amazon

I am a Senior Applied Scientist in AGI Foundations at Amazon, California. My research focuses on advancing Artificial Intelligence through the development of large language models, agentic models, and reasoning models that are helpful, capable, and safe. My specific interests include benchmark curation, the design of robust evaluation metrics, and the evaluation of models to assess their alignment with responsible AI policies. I am also engaged in uncovering model vulnerabilities through novel jailbreak attacks and red-teaming methodologies. I am interested in developing agentic systems and LLMs for applications such as healthcare and others.

Prior to joining Amazon, I completed my Ph.D. in Computing and Information Sciences at the Rochester Institute of Technology (RIT), where I worked under the supervision of Dr. Linwei Wang in the Computational Biomedicine Lab. My doctoral research centered on personalization and uncertainty quantification in multi-scale 3D simulation models of cardiac electrophysiology. This work allowed me to operate at the intersection of machine learning—specifically Bayesian modeling, optimization, generative modeling, and graph convolutional networks—and computational healthcare, with a focus on personalized cardiac modeling.

I am always open to research collaborations in areas related to AI safety, model evaluation, and trustworthy machine learning. Feel free to reach out at jwala [dot] dhamala [at] gmail [dot] com if you are interested in collaborating.


Research Interests

Agentic AI Systems. I am interested in building capable, safe, and reliable agentic and reasoning models—large language models and multi-agent systems that can operate autonomously while remaining aligned with responsible AI policies. [Amazon Nova] [Agentic Benchmarks] [Tree-of-Traversals]

Evaluation & Benchmarking. I design rigorous benchmarks and robust evaluation metrics for assessing model capabilities, safety alignment, and responsible AI compliance across language generation and agentic tasks. [BOLD] [TANGO] [Intrinsic vs Extrinsic Fairness]

Discovering Capabilities & Limitations. I probe for emergent model behaviors through red-teaming, jailbreak attacks, and adversarial methods, uncovering vulnerabilities such as deception in long-horizon interactions and biases in open-ended generation. [LH-Deception] [Intrinsic vs Extrinsic Fairness] [Resolving Ambiguities]

AI for Healthcare. I apply AI to computational healthcare problems, from personalized cardiac modeling and uncertainty quantification to exploring agentic AI applications in clinical decision support. [MedIA 2020] [MICCAI 2019] [MICCAI 2018]


Selected Publications

For a comprehensive list of my publications, please visit my Google Scholar profile.


  • LH-Deception: Simulating and Understanding LLM Deceptive Behaviors in Long-Horizon Interactions
    Y. Xu, X. Zhang, M. Yeh, J. Dhamala, O. A. Dia, R. Gupta, Y. Li
    ICLR, 2026
  • The Amazon Nova Family of Models: Technical Report and Model Card
    Amazon AGI (incl. J. Dhamala)
    arXiv, 2025
  • Establishing Best Practices for Building Rigorous Agentic Benchmarks
    Y. Zhu, T. Jin, Y. Pruksachatkun, A. Zhang, S. Liu, S. Cui, S. Kapoor, S. Longpre, K. Meng, R. Weiss, F. Barez, R. Gupta, J. Dhamala, J. Merizian, M. Giulianelli, H. Coppock, C. Ududec, J. Sekhon, J. Steinhardt, A. Kellerman, S. Schwettmann, M. Zaharia, I. Stoica, P. Liang, D. Kang
    NeurIPS Datasets and Benchmarks Track, 2025
  • MICo: Preventative Detoxification of Large Language Models through Inhibition Control
    R. Siegelmann, N. Mehrabi, P. Goyal, L. Bauer, J. Dhamala, A. Galstyan, R. Gupta, R. Ghanadan
    NAACL Findings, 2024
  • Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies
    A. Ovalle, N. Mehrabi, P. Goyal, J. Dhamala, K. Chang, A. Galstyan, R. Zemel, Y. Pinter, R. Gupta
    NAACL Findings, 2024
  • Tree-of-Traversals: A Zero-Shot Reasoning Algorithm for Augmenting Black-box Language Models with Knowledge Graphs
    E. Markowitz, A. Ramakrishna, J. Dhamala, N. Mehrabi, C. Peris, R. Gupta, K. Chang, A. Galstyan
    ACL, 2024
  • “I’m fully who I am”: Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation
    A. Ovalle, P. Goyal, J. Dhamala, Z. Jaggers, K. Chang, A. Galstyan, R. Zemel, R. Gupta
    FAccT, 2023
  • Resolving Ambiguities in Text-to-Image Generative Models
    N. Mehrabi, P. Goyal, A. Verma, J. Dhamala, V. Kumar, Q. Hu, K. Chang, R. Zemel, A. Galstyan, R. Gupta
    ACL, 2023
  • Multi-VALUE: A Framework for Cross-Dialectal English NLP
    C. Ziems, W. Held, J. Yang, J. Dhamala, R. Gupta, D. Yang
    ACL, 2023
  • Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal
    U. Gupta, J. Dhamala, V. Kumar, A. Verma, Y. Pruksachatkun, S. Krishna, R. Gupta, K. Chang, G. Steeg, A. Galstyan
    ACL Findings, 2022
  • On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations
    Y. Trista Cao, Y. Pruksachatkun, K. Chang, R. Gupta, V. Kumar, J. Dhamala, A. Galstyan
    ACL, 2022
  • BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation
    J. Dhamala, T. Sun, V. Kumar, S. Krishna, Y. Pruksachatkun, K. Chang, R. Gupta
    ACM FAccT, 2021
Earlier Publications (2020 and before — PhD research)
  • Embedding High-dimensional Bayesian Optimization via Generative Modeling: Parameter Personalization of Cardiac Electrophysiological Models
    J. Dhamala, H. J. Arevalo, J. L. Sapp, M. Horacek, K. C. Wu, N. A. Trayanova, L. Wang
    Medical Image Analysis (MedIA), 2020
  • Bayesian Optimization on Large Graphs via a Graph Convolutional Generative Model: Application in Cardiac Model Personalization
    J. Dhamala, J. L. Sapp, M. Horacek, L. Wang
    MICCAI, 2019
  • High-dimensional Bayesian Optimization of Personalized Cardiac Model Parameters via an Embedded Generative Model
    J. Dhamala, J. L. Sapp, M. Horacek, L. Wang
    MICCAI, 2018
  • Quantifying the Uncertainty in Model Parameters using Gaussian Process-Based Markov Chain Monte Carlo in Cardiac Electrophysiology
    J. Dhamala, H. J. Arevalo, J. L. Sapp, M. Horacek, K. C. Wu, N. A. Trayanova, L. Wang
    Medical Image Analysis (MedIA), 2018
  • Multivariate Time-series Similarity Assessment via Unsupervised Representation Learning and Stratified Locality Sensitive Hashing: Application to Early Acute Hypotensive Episode Detection
    J. Dhamala, E. Azuh, A. Al-Dujaili, J. Rubin, U. O'Reilly
    IEEE Sensors Letters, 2018; NeurIPS ML4H Workshop, 2018
  • Quantifying the Uncertainty in Model Parameters using Gaussian Process-Based Markov Chain Monte Carlo: An Application to Cardiac Electrophysiological Models
    J. Dhamala, J. L. Sapp, M. Horacek, L. Wang
    IPMI, 2017
  • Spatially-Adaptive Multi-Scale Optimization for Local Parameter Estimation in Cardiac Electrophysiology
    J. Dhamala, H. J. Arevalo, J. L. Sapp, M. Horacek, K. C. Wu, N. A. Trayanova, L. Wang
    IEEE Transactions on Medical Imaging (TMI), 2017
  • Spatially-Adaptive Multi-scale Optimization for Local Parameter Estimation: Application in Cardiac Electrophysiological Models
    J. Dhamala, J. L. Sapp, M. Horacek, L. Wang
    MICCAI, 2016

News

  • July 2026 NEW Co-organizing the 6th TrustNLP Workshop at ACL 2026 in San Diego.
  • 2026 NEW LH-Deception on LLM deceptive behaviors in long-horizon interactions accepted at ICLR 2026.
  • Feb 2025 Serving as Area Chair for ACL Rolling Review (ARR).
  • 2025 Paper on best practices for agentic benchmarks accepted at NeurIPS 2025 Datasets and Benchmarks Track.
  • 2025 Released the Amazon Nova Technical Report.
  • May 2025 Organized TrustNLP workshop at NAACL 2025.
  • 2024 Tree-of-Traversals — paper led by our intern Elan on zero-shot reasoning with knowledge graphs accepted at ACL.
Earlier News
  • 2023Paper led by our intern Nina accepted at ACL.
  • 2023Paper led by our intern Elia accepted at ACM FAccT 2023.
  • May 2022Three papers accepted at ACL 2022: [1] [2] [3].
  • May 2022Organized TrustNLP workshop at NAACL 2022.
  • July 2021Organized Responsible AI workshop at KDD 2021.
  • May 2021Organized TrustNLP at NAACL 2021.
  • Jan 2021Paper on bias in open-ended generation accepted at ACM FAccT. Dataset: BOLD.
  • Dec 2020Panelist on AI fairness discussion at NeurIPS 2020: Watch here.
  • Oct 2020Paper with intern Ansel accepted at EMNLP workshop.
  • Feb 2020Paper accepted at Medical Image Analysis.
  • Feb 2020Successfully defended PhD: Thesis.
  • Dec 2019Joined Amazon Alexa NU-AI as Research Scientist.
  • Jun 2019Paper finalist for MICCAI Young Scientist Award.
  • Oct 2018Paper accepted at IEEE Sensors Letters and NeurIPS ML4H.
  • Sep 2018Paper finalist for MICCAI 2018 Young Scientist Award.

Service & Activities

RoleVenueYear
Co-organizer TrustNLP Workshop — ACL & NAACL 2021–2026
Area Chair ACL Rolling Review (ARR) 2025
Reviewer ACL Rolling Review (ARR) 2024–2026
Co-organizer Responsible AI Workshop — KDD 2021
Student Co-organizer Hackathon on PVC, Consortium of ECG Imaging 2015–2017
Student Co-organizer Pre-orientation Program, Women in Computing, RIT 2018