AI Research

Curating Pioneering Research

LMDX: Language Model-based Document Information Extraction and Localization

/Recents:

Prof. Otto NomosOct 04, 2023 ∙ 2 min read

LMDX: Language Model-based Document Information Extraction and Localization

Abstract Commentary & Rating

/LLM

/Transformers

Cure the headache of Transformers via Collinear Constrained Attention
Cure the headache of Transformers via Collinear Constrained Attention
Prof. Otto NomosOct 04, 2023 ∙ 2 min read
Abstract Commentary & Rating
Uncovering mesa-optimization algorithms in Transformers
Uncovering mesa-optimization algorithms in Transformers
Prof. Otto NomosOct 03, 2023 ∙ 2 min read
Abstract Commentary & Rating
Gated recurrent neural networks discover attention
Gated recurrent neural networks discover attention
Prof. Otto NomosOct 03, 2023 ∙ 2 min read
Abstract Commentary & Rating
One Wide Feedforward is All You Need
One Wide Feedforward is All You Need
Prof. Otto NomosOct 03, 2023 ∙ 2 min read
Abstract Commentary & Rating
Vector Search with OpenAI Embeddings: Lucene Is All You Need
Vector Search with OpenAI Embeddings: Lucene Is All You Need
Prof. Otto NomosOct 02, 2023 ∙ 2 min read
Abstract Commentary & Rating
Bayesian Flow Networks
Bayesian Flow Networks
Prof. Otto NomosOct 02, 2023 ∙ 2 min read
Abstract Commentary & Rating
YaRN: Efficient Context Window Extension of Large Language Models
YaRN: Efficient Context Window Extension of Large Language Models
Prof. Otto NomosOct 02, 2023 ∙ 2 min read
Abstract Commentary & Rating
LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models
LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models
Prof. Otto NomosOct 02, 2023 ∙ 2 min read
Abstract Commentary & Rating
Composable Function-preserving Expansions for Transformer Architectures
Composable Function-preserving Expansions for Transformer Architectures
Prof. Otto NomosOct 02, 2023 ∙ 2 min read
Abstract Commentary & Rating

/Audio

/Image

/Retrieval

RAVEN: In-Context Learning with Retrieval Augmented Encoder-Decoder Language Models
RAVEN: In-Context Learning with Retrieval Augmented Encoder-Decoder Language Models
Prof. Otto NomosOct 02, 2023 ∙ 2 min read
Abstract Commentary & Rating

/Diffusion

PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
Prof. Otto NomosOct 02, 2023 ∙ 2 min read
Abstract Commentary & Rating

Subscribe For The Latest Updates Subscribe to the newsletter and never miss the new post every week.

AI Research

LMDX: Language Model-based Document Information Extraction and Localization

Chain-of-Verification Reduces Hallucination in Large Language Models

Chain-of-Verification Reduces Hallucination in Large Language Models

Stabilizing RLHF through Advantage Model and Selective Rehearsal

Stabilizing RLHF through Advantage Model and Selective Rehearsal

SlimPajama-DC: Understanding Data Combinations for LLM Training

SlimPajama-DC: Understanding Data Combinations for LLM Training

Baichuan 2: Open Large-scale Language Models

Baichuan 2: Open Large-scale Language Models

OpenBA: An Open-sourced 15B Bilingual Asymmetric seq2seq Model Pre-trained from Scratch

OpenBA: An Open-sourced 15B Bilingual Asymmetric seq2seq Model Pre-trained from Scratch

Language Modeling Is Compression

Language Modeling Is Compression

Augmenting text for spoken language understanding with Large Language Models

Augmenting text for spoken language understanding with Large Language Models

S3-DST: Structured Open-Domain Dialogue Segmentation and State Tracking in the Era of LLMs

S3-DST: Structured Open-Domain Dialogue Segmentation and State Tracking in the Era of LLMs

Recovering from Privacy-Preserving Masking with Large Language Models

Recovering from Privacy-Preserving Masking with Large Language Models

Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data?

Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data?

MindAgent: Emergent Gaming Interaction

MindAgent: Emergent Gaming Interaction

An Empirical Study of Scaling Instruct-Tuned Large Multimodal Models

An Empirical Study of Scaling Instruct-Tuned Large Multimodal Models

Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference Using Sorted Fine-Tuning (SoFT)

Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference Using Sorted Fine-Tuning (SoFT)

Adapting Large Language Models via Reading Comprehension

Adapting Large Language Models via Reading Comprehension

PDFTriage: Question Answering over Long, Structured Documents

PDFTriage: Question Answering over Long, Structured Documents

Contrastive Decoding Improves Reasoning in Large Language Models

Contrastive Decoding Improves Reasoning in Large Language Models

CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages

CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages

A Data Source for Reasoning Embodied Agents

A Data Source for Reasoning Embodied Agents

Leveraging Contextual Information for Effective Entity Salience Detection

Leveraging Contextual Information for Effective Entity Salience Detection

LASER: LLM Agent with State-Space Exploration for Web Navigation

LASER: LLM Agent with State-Space Exploration for Web Navigation

Sparse Autoencoders Find Highly Interpretable Features in Language Models

Sparse Autoencoders Find Highly Interpretable Features in Language Models

Investigating Answerability of LLMs for Long-Form Question Answering

Investigating Answerability of LLMs for Long-Form Question Answering

Scaling Laws for Sparsely-Connected Foundation Models

Scaling Laws for Sparsely-Connected Foundation Models

Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers

Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers

Ambiguity-Aware In-Context Learning with Large Language Models

Ambiguity-Aware In-Context Learning with Large Language Models

Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?

Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?

Clinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts

Clinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts

Agents: An Open-source Framework for Autonomous Language Agents

Agents: An Open-source Framework for Autonomous Language Agents

Statistical Rejection Sampling Improves Preference Optimization

Statistical Rejection Sampling Improves Preference Optimization

Large Language Models for Compiler Optimization

Large Language Models for Compiler Optimization

AstroLLaMA: Towards Specialized Foundation Models in Astronomy

AstroLLaMA: Towards Specialized Foundation Models in Astronomy

Large Language Model for Science: A Study on P vs. NP

Large Language Model for Science: A Study on P vs. NP

Efficient Memory Management for Large Language Model Serving with PagedAttention

Efficient Memory Management for Large Language Model Serving with PagedAttention

FIAT: Fusing learning paradigms with Instruction-Accelerated Tuning

FIAT: Fusing learning paradigms with Instruction-Accelerated Tuning

Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs

Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs

When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale

When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale

Neurons in Large Language Models: Dead, N-gram, Positional

Neurons in Large Language Models: Dead, N-gram, Positional

Textbooks Are All You Need II: phi-1.5 technical report

Textbooks Are All You Need II: phi-1.5 technical report

DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphs

DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphs