I am a research scientist working on Responsible AI and the evaluation of large language models. I am interested in how we measure whether AI systems are safe, fair, and reliable: designing benchmarks and metrics, building trustworthy automated evaluators such as LLM judges, and uncovering emerging risks, from deception and long-context safety to failures that arise when multiple AI agents interact. My broader goal is evaluation science that turns measurements into insights that guide how models are built and deployed. I am currently a Senior Applied Scientist at Amazon, and I received my Ph.D. in Computing and Information Sciences from the Rochester Institute of Technology under Dr. Linwei Wang.

I am open to collaborations on AI safety, model evaluation, and trustworthy ML. Reach me at jwala [dot] dhamala [at] gmail [dot] com.

Research Interests

Evaluation & Benchmarks. Rigorous benchmarks, metrics, and automated evaluators (LLM-as-a-judge and rule-based) for measuring safety, fairness, and harmful content in LLMs. [BOLD] [Intrinsic vs Extrinsic Fairness] [Agentic Benchmarks] [Amazon Nova]

Emerging Risks & Failure Discovery. Red-teaming and adversarial probing to surface failures that standard evaluations miss, including LLM deception, long-context safety, and emerging risks in agentic and multi-agent systems. [LH-Deception] [JAB]

AI for Healthcare. Reliable, uncertainty-aware machine learning for high-stakes clinical and biomedical applications, building on my Ph.D. work on probabilistic ML (Bayesian optimization, Gaussian processes, generative models) for personalizing cardiac simulation models. [Graph BO-VAE] [GP-MCMC] [S2S-LSH]

Earlier, I also worked on fairness mitigation for language understanding models using weakly supervised learning, active learning, and data augmentation.

Highlights

• Co-organizer, TrustNLP Workshop (ACL & NAACL, 2021–2026)
• Area Chair, ACL Rolling Review (2025)
• Two-time finalist, MICCAI Young Scientist Award (2018, 2019)

Recent News

• July 2026: Co-organizing the 6th TrustNLP Workshop at ACL 2026 in San Diego.
• 2026: LH-Deception on LLM deceptive behaviors in long-horizon interactions accepted at ICLR 2026.
• 2025: Paper on best practices for agentic benchmarks accepted at NeurIPS 2025 Datasets and Benchmarks Track.
• 2025: Released the Amazon Nova Technical Report.

See all news for earlier updates. Full publication list on the Publications page, Google Scholar, and in my CV.