01
SCREWS: Reasoning with Revisions
A framework that explores how reasoning can be improved in LLMs by iteratively refining the outputs
I am a third year Ph.D. student at ETH Zürich in Switzerland, supervised by Prof. Mrinmaya Sachan and Nicholas Monath from Google DeepMind.
My research and professional interests are centered around Natural Language Processing and Machine Learning
in general, especially towards understanding the reasoning capabilities of LLMs. My PhD research investigates how generative
LLMs reason, what factors influence their reasoning performance., and how to optimize their reasoning abilities.
During my PhD, I spent my summers as an intern at FAIR, Meta with Asli Celikyilmaz on agent-based
reasoning, at Microsoft Research with Patrick Xia and Jason Eisner
on improving the reasoning capabilities of LLMs by revising their output, and
at Amazon Alexa AI on setting constraints on the output space of a neural network.
I have experience with fine-tuning LLMs at scale (up to LLAMA 70B), distilling reasoning capabilities from
larger models (GPT-4 and chatGPT) to smaller models (LLAMA 7B, T5, GPT-2), and aligning LLMs to generate more accurate
and contextually appropriate responses (PPO, RLHF).
Before starting my PhD, I gained valuable experience in researching and deploying conversational AI systems at Insiders Technology GmbH
and NeuralSpace .
I also worked with Prof. Marcus Liwicki and Prof.
Seiichi Uchida during my masters studies.
In my free time, I enjoy playing tennis, reading about conspiracy theories, and collecting sneakers.
A framework that explores how reasoning can be improved in LLMs by iteratively refining the outputs
Can reasoning be distilled from an larger model to a smaller model?
Can decomposition of problem into simpler problems assist students in learning a concept better?
A Causal Framework to quantify the robustness of mathematical reasoning with language models
An adversarial training strategy to achieve information-based stochastic dropout to train sequence VAEs.
LongtoNotes is a corpus of coreference-annotated documents which are up to 8x the length of documents in Ontonotes.