-
AI EvaluationSora: OpenAI’s Text-to-Video Generation Model
Introduction This week, we talk about the implications of Text-to-Video Generation and speculate as to the possibilities (and limitations) of this incredible technology with some hot takes. Dat… Sarah Welsh March 1, 2024 37 min read -
AI EvaluationWhat Does It Take To Pioneer Successful LLM Applications In Healthcare and the Life Sciences?
Peter Leimbigler is a Data Science Team Leader within the Consulting practice at Klick Health. As the largest independent commercialization partner in its industry, Klick pioneers new AI-powered… David Burch February 21, 2024 11 min read -
AI EvaluationPhi-2 Model
Introduction With only 2.7 billion parameters, Phi-2 surpasses the performance of Mistral and Llama-2 models at 7B and 13B parameters on various aggregated benchmarks. Notably, it achieves better… Sarah Welsh January 31, 2024 36 min read -
AI EvaluationEvaluate RAG with LLM Evals and Benchmarking
Recently, I attended a workshop organized by Arize AI titled “RAG Time! Evaluate RAG with LLM Evals and Benchmarking.” Hosted by Amber Roberts – ML Growth Lead at… Joel Bowman January 1, 2024 13 min read -
AI EvaluationMistral AI (Mixtral-8x7B): Performance, Benchmarks
Introduction For the last paper read of the year, Arize CPO & Co-Founder, Aparna Dhinakaran, is joined by a Dat Ngo (ML Solutions Architect) and Aman Khan (Group… Sarah Welsh December 27, 2023 35 min read -
AI EvaluationThe Geometry of Truth: Emergent Linear Structure in LLM Representation of True/False Datasets
Introduction For this paper read, we’re joined by Samuel Marks, Postdoctoral Research Associate at Northeastern University, to discuss his paper, “The Geometry of Truth: Emergent Linear Structure in… Sarah Welsh November 14, 2023 32 min read -
AI EvaluationIngesting Data for Semantic Searches in a Production-Ready Way
The current ecosystem around LLMs, semantic search and vector storage makes it easy to prototype but difficult to move into production. Ingesting large volumes of data specifically for… David Garnitz November 8, 2023 10 min read -
AI EvaluationTowards Monosemanticity: Decomposing Language Models With Dictionary Learning
Introduction In this paper read, we discuss “Towards Monosemanticity: Decomposing Language Models With Dictionary Learning,” a paper from Anthropic that addresses the challenge of understanding the inner workings… Sarah Welsh November 2, 2023 26 min read -
AI EvaluationLarge Content And Behavior Models to Understand, Simulate, and Optimize Content and Behavior.
Introduction Amber Roberts and Sally-Ann DeLucia discuss “Large Content And Behavior Models To Understand, Simulate, And Optimize Content And Behavior.” This paper highlights that while LLMs have great… Sarah Welsh September 18, 2023 36 min read -
AI EvaluationExtending the Context Window of LLaMA Models Paper Reading
Introduction During this week’s paper reading event, we are thrilled to announce that we will be joined by Frank Liu, Director of Operations, and ML Architect at Zilliz,… Sarah Welsh August 7, 2023 32 min read -
AI EvaluationOrca: Progressive Learning from Complex Explanation Traces of GPT-4 Paper Reading
Introduction Recent research focuses on improving smaller models through imitation learning using outputs from large foundation models (LFMs). Challenges include limited imitation signals, homogeneous training data, and a… Sarah Welsh July 13, 2023 30 min read -
AI Evaluation
One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuning
Introduction In this week’s paper reading, we discuss “One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuning.” GLoRA is a universal, parameter-efficient fine-tuning approach for diverse tasks. It enhances LoRA with… Sarah Welsh July 3, 2023 32 min read -
AI Evaluation
HyDE: Precise Zero-Shot Dense Retrieval without Relevance Labels
Introduction In this paper reading, we explore HyDE: Precise Zero-Shot Dense Retrieval without Relevance Labels. HyDE is a thrilling zero-shot learning technique that combines GPT-3’s language understanding with… Sarah Welsh June 27, 2023 30 min read -
AI Evaluation
LoRA: Low-Rank Adaptation of Large Language Models Paper Reading and Discussion
Introduction In this paper reading, we discuss LoRA, which freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly… Sarah Welsh June 12, 2023 28 min read -
AI Evaluation
Retrieval-Augmented Generation – Paper Reading and Discussion
Introduction In this paper reading, we discuss “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” We know GPT-like LLMs are great at soaking up knowledge during pre-training and fine-tuning them… Sarah Welsh June 9, 2023 34 min read
Don’t ship vibes.
Arize gives AI teams observability and evals to understand and improve agent performance.
