LLM benchmarks, agent evaluations, and experiments.
-
ResearchReinforcement Learning in the Era of LLMs
Introduction This week, we explore Reinforcement Learning in the Era of LLMs: What is Essential? What is needed? An RL Perspective on RLHF, Prompting, and Beyond, with Claire… Sarah Welsh March 15, 2024 38 min read -
ResearchSora: OpenAI’s Text-to-Video Generation Model
Introduction This week, we talk about the implications of Text-to-Video Generation and speculate as to the possibilities (and limitations) of this incredible technology with some hot takes. Dat… Sarah Welsh March 1, 2024 37 min read -
ResearchEvaluating the Generation Stage in RAG
In retrieval-augmented generation (RAG), retrieval often steals the spotlight, while the generation stage receives less attention. To address this gap, we conducted a series of tests to see… Aparna Dhinakaran February 15, 2024 4 min read -
ResearchRAG vs Fine-Tuning
Introduction This week we discussed “RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture.” This paper that explores a pipeline for Fine-tuning and RAG, and presents… Sarah Welsh February 8, 2024 31 min read -
ResearchPhi-2 Model
Introduction With only 2.7 billion parameters, Phi-2 surpasses the performance of Mistral and Llama-2 models at 7B and 13B parameters on various aggregated benchmarks. Notably, it achieves better… Sarah Welsh January 31, 2024 36 min read -
ResearchMistral AI (Mixtral-8x7B): Performance, Benchmarks
Introduction For the last paper read of the year, Arize CPO & Co-Founder, Aparna Dhinakaran, is joined by a Dat Ngo (ML Solutions Architect) and Aman Khan (Group… Sarah Welsh December 27, 2023 35 min read -
ResearchHow to Prompt LLMs for Text-to-SQL
Introduction For this paper read, we’re joined by Shuaichen Chang, now an Applied Scientist at AWS AI Lab and author of this week’s paper to discuss his findings.… Sarah Welsh December 18, 2023 28 min read -
ResearchThe Geometry of Truth: Emergent Linear Structure in LLM Representation of True/False Datasets
Introduction For this paper read, we’re joined by Samuel Marks, Postdoctoral Research Associate at Northeastern University, to discuss his paper, “The Geometry of Truth: Emergent Linear Structure in… Sarah Welsh November 14, 2023 32 min read -
ResearchTowards Monosemanticity: Decomposing Language Models With Dictionary Learning
Introduction In this paper read, we discuss “Towards Monosemanticity: Decomposing Language Models With Dictionary Learning,” a paper from Anthropic that addresses the challenge of understanding the inner workings… Sarah Welsh November 2, 2023 26 min read -
ResearchRankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models
Introduction In this paper reading, we’ll be discussing RankVicuna, the first fully open-source LLM capable of performing high-quality listwise reranking in a zero-shot setting. While researchers have successfully… Sarah Welsh October 17, 2023 32 min read -
ResearchExplaining Grokking Through Circuit Efficiency
Introduction Join Arize Co-Founder & CEO Jason Lopatecki, and ML Solutions Engineer, Sally-Ann DeLucia, as they discuss “Explaining Grokking Through Circuit Efficiency.” This paper explores novel predictions about… Sarah Welsh October 6, 2023 27 min read -
ResearchLarge Content And Behavior Models to Understand, Simulate, and Optimize Content and Behavior.
Introduction Amber Roberts and Sally-Ann DeLucia discuss “Large Content And Behavior Models To Understand, Simulate, And Optimize Content And Behavior.” This paper highlights that while LLMs have great… Sarah Welsh September 18, 2023 36 min read -
ResearchSkeleton of Thought: LLMs Can Do Parallel Decoding Paper Reading
Introduction Join us for an exploration of the ‘Skeleton-of-Thought’ (SoT) approach, aimed at reducing large language model latency while enhancing answer quality, with the presence of two authors,… Sarah Welsh August 24, 2023 28 min read -
ResearchExtending the Context Window of LLaMA Models Paper Reading
Introduction During this week’s paper reading event, we are thrilled to announce that we will be joined by Frank Liu, Director of Operations, and ML Architect at Zilliz,… Sarah Welsh August 7, 2023 32 min read -
ResearchLlama 2: Open Foundation and Fine-Tuned Chat Models Paper Reading
Introduction In this paper reading, we explore the paper “Llama 2: Open Foundation and Fine-Tuned Chat Models.” The paper introduces Llama 2, a collection of pretrained and fine-tuned… Sarah Welsh August 4, 2023 22 min read
Don’t ship vibes.
Arize gives AI teams observability and evals to understand and improve agent performance.
