I am a research scientist working on Responsible AI, AI safety, and the evaluation of large language models and generative AI systems. I am interested in how we measure whether AI systems are safe, fair, and reliable: designing benchmarks and metrics, building trustworthy automated evaluators such as LLM judges, and uncovering emerging risks, from deception and long-context safety to failures that arise when multiple AI agents interact. My goal is evaluation rigorous enough to guide how models are built and deployed. I am currently a Senior Applied Scientist at Amazon, where I recently led Responsible AI quantitative evaluation for Amazon Nova 1.0 and Nova 2.0 (AI Service Cards: Nova 1.0, Nova 2 Lite). I received my Ph.D. in Computing and Information Sciences from the Rochester Institute of Technology in 2020 under Dr. Linwei Wang.

I am open to research collaborations, speaking, and advising on AI safety, model evaluation, and trustworthy ML. Reach me at jwala [dot] dhamala [at] gmail [dot] com.

Research Interests

Evaluation & Benchmarks. Rigorous benchmarks, metrics, and automated evaluators (LLM-as-a-judge and rule-based) for measuring safety, fairness, and content safety in LLMs, including multilingual and low-resource settings. [BOLD, ACM FAccT 2021] [Intrinsic vs Extrinsic Fairness, ACL 2022] [Agentic Benchmarks, NeurIPS D&B 2025] [Amazon Nova, arXiv 2025]

Emerging Risks & Failure Discovery. Red-teaming, jailbreaks, and adversarial probing to surface failures that standard evaluations miss, including LLM deception, long-context safety, and risks in agentic and multi-agent systems. [LH-Deception, ICLR 2026] [DECOR, arXiv 2026] [JAB, arXiv 2023]

Trustworthy AI for Healthcare. Evaluation, uncertainty quantification, and reliability for AI in high-stakes clinical settings. My Ph.D. developed probabilistic ML (Bayesian optimization, Gaussian processes, generative models) for personalized cardiac simulation, and I am interested in bringing rigorous safety evaluation to clinical and health LLMs. [Graph BO-VAE, MICCAI 2019] [GP-MCMC, MedIA 2018] [S2S-LSH, IEEE Sensors Letters 2018]

Earlier, I worked on fairness mitigation for language understanding models using weakly supervised learning, active learning, and data augmentation.

Highlights

• Responsible AI quantitative evaluation lead, Amazon Nova 1.0 and 2.0: Technical Report (2025), AI Service Cards for Nova 1.0 and Nova 2 Lite
• BOLD, ACM FAccT 2021: widely used benchmark for bias in open-ended language generation
• Co-organizer, TrustNLP Workshop (ACL & NAACL, 2021–2026) [Retrospective]
• Area Chair, ACL Rolling Review (2025)
• Two-time finalist, MICCAI Young Scientist Award (2018, 2019)

Recent News

• Jul 2026: Co-organized the 6th TrustNLP Workshop at ACL 2026 in San Diego.
• 2026: LH-Deception (ICLR 2026), on LLM deceptive behaviors in long-horizon interactions, in collaboration with Prof. Sharon Li’s group at UW–Madison, accepted.

See all news for earlier updates. Full publication list on the Publications page, Google Scholar, and in my CV.