Biography

Hi, This is Nikhil Madaan. I’m a Senior AI Research Engineer at Bloomberg AI, focusing on multi-variate time-series modeling to build real-time pricing models.

Previously, I was a M.S. in Computer Engineering (AI-ML Concentration) student at Carnegie Mellon University. During my time at CMU, I was a part of MultiComp Lab, where I worked with Jianing “Jed” Yang and Prof. Louis-Philippe Morency on bias analysis in Multimodal QA datasets. I was also a part of Lion’s Research group, where I worked on Personalized Federated Learning with Dr. Taejin Kim and Prof. Carlee Joe-Wong.

During my master’s, I interned at Amazon as Applied Scientist, with the Media and ADs group, where I worked with Dr. Manisha Verma on multi-modal product headline generation. Prior to my master’s I worked as SWE-ML at Flipkart as a part of the Catalog Ingestion team, where I played a significant role in designing and implementing scalable AI systems.

I am particularly interested in the applications of Deep Learning in areas such as Natural Language Processing, 3d-Computer Vision, and Multimodal ML.

Education

Carnegie Mellon University Carnegie Mellon University
Aug 2021 - Dec 2022

M.S. in Computer Engineering (AI-ML Concentration)


Work Experience

Bloomberg AI Bloomberg AI
Jan 2023 - Present

Senior AI Researcher

  • Working on leveraging ML models for multi-variate time-series modeling to build real-time pricing models for Fixed-Income securities.

Amazon Amazon - Media and Ads Group
May 2022 - Aug 2022

Applied Scientist Intern, Advisor: Dr. Manisha Verma

  • Worked on generating headlines for products by factoring in multiple modalities such as Product Images and product attributes (Text); using SOTA multimodal fusion networks such as Flava, Mantis.
  • Employed contrastive learning to improve the diversity of the generated headlines and rouge, bleu score by 53.5% and 145% respectively, w.r.t unimodal models.

MultiComp Lab - LTI, CMU MultiComp Lab
Jan 2022 - May 2023

Graduate Research Assistant, Advisor: Prof. L.P. Morency , Mentor: Jianing “Jed” Yang

  • Worked on analyzing QA bias in Multi-modal Question Answering Systems using fine-tuned language models.

Lions Research Lab Lions Research Lab
Jan 2022 - Dec 2022

Graduate Research Assistant, Advisor: Prof. Carlee Joe-Wong

  • Worked on the transferability of adversarial attacks in Personalized Federated Learning setup and trying to make the learnt model more robust to such attacks.

Flipkart Walmart Grp (Flipkart)
Jul 2019 - Aug 2021

Software Engineer - Machine Learning

  • Leveraged Image encoders such as ViT, ResNets, to generate embeddings of the product images. Indexed the generated embeddings, added multi-cluster support for reads and writes, and used the embeddings to support Product Deduplication.

  • Developed a prioritized distributed message processing xtension to the camel-Kafka component in Java, to support priority consumption of records and implemented various consumption strategies to support different use cases.


Publications

MultiComp Lab - LTI, CMU

LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agent
Jianing (Jed) Yang, Xuweiyi Chen, Shengyi Qian, Nikhil Madaan, Madhavan Iyengar, David Fouhey, Joyce Y. Chai
Accepted at ICRA 2024 | LangRob @ CoRL(LangRob) 2023
[website] | [pdf] | [code] | [video]



MultiComp Lab - LTI, CMU

Contrastive Multimodal Text Generation for E-Commerce Brand Advertising
Nikhil Madaan *, Krishna Reddy Kesari, Manisha Verma, Shaunak Mishra, Tor Steiner
Accepted at KDD (Multimodal) 2023
[pdf]



Characterizing Internal Evasion Attacks in Federated Learning

Adversarial Robustness Unhardening via Backdoor Attacks in Federated Learning
Taejin Kim, Jiarui Li, Nikhil Madaan, Shubranshu Singh,Carlee Joe-Wong
Accepted at Neurips 2023
[pdf]



Characterizing Internal Evasion Attacks in Federated Learning

INVESTIGATING THE IMPORTANT TEMPORAL MODULATIONS FOR DEEP-LEARNING-BASED SPEECH ACTIVITY DETECTION
Tyler Vuong, Nikhil Madaan, Rohan Panda, Richard M. Stern
Accepted at SLT 2022
[pdf]



Characterizing Internal Evasion Attacks in Federated Learning

Characterizing Internal Evasion Attacks in Federated Learning
Taejin Kim, Shubranshu Singh, Nikhil Madaan, Carlee Joe-Wong
Accepted at AISTATS 2023
[pdf]


Reviewing

Sub-Reviewer CIKM,22