The Evaluator
Your go-to blog for insights on AI observability and evaluation.
Showing 291–300 of 458 posts (page 30 of 46)
Explaining Grokking Through Circuit Efficiency
Introduction Join Arize Co-Founder & CEO Jason Lopatecki, and ML Solutions Engineer, Sally-Ann DeLucia, as they discuss “Explaining Grokking Through Circuit Efficiency.” This paper explores novel predictions about grokking, providing significant evidence in favor of its explanation. Most strikingly, the research conducted in this paper demonstrates two novel and surprising behaviors: ungrokking, in which a…
LLM Tracing and Observability
What is LLM App Tracing? The rise of large language model (LLM) application development has enabled developers to move quickly in building applications powered by LLMs. The abstractions created by these frameworks can accelerate development, but also make it hard to debug an LLM app. This is where Arize Phoenix, a popular open-source library for…
Arize AI Debuts Integration with Anyscale Endpoints
At Ray Summit 2023, Anyscale Endpoints – a new service enabling developers to integrate fast, cost-efficient, and scalable large language models (LLMs) into their applications using popular LLM APIs – made its grand debut. The service promises to be “the fastest way to fine-tune and deploy powerful open-source LLMs at scale.” In addition to Hugging…
Sign up for our newsletter, The Evaluator — and stay in the know with updates and new resources:
Large Content And Behavior Models to Understand, Simulate, and Optimize Content and Behavior.
Introduction Amber Roberts and Sally-Ann DeLucia discuss “Large Content And Behavior Models To Understand, Simulate, And Optimize Content And Behavior.” This paper highlights that while LLMs have great generalization capabilities, they struggle to effectively predict and optimize communication to get the desired receiver behavior. We’ll explore whether this might be because of a lack of…
Skeleton of Thought: LLMs Can Do Parallel Decoding Paper Reading
Introduction Join us for an exploration of the ‘Skeleton-of-Thought’ (SoT) approach, aimed at reducing large language model latency while enhancing answer quality, with the presence of two authors, Xuefei Ning and Zinan Lin. SoT’s innovative methodology guides LLMs to construct answer skeletons before parallel content elaboration, achieving impressive speed-ups of up to 2.39x across 11…
How To Thrive During Your First Tech Internship: What I Learned Interning at a Rapidly-Growing LLMOps Startup
SWE, AI, ML, FAANG, startups — with all the coveted tech jargon you’re bound to hear during recruiting season, it’s easy to get bombarded and discouraged. I know I was. As a junior in college, I was in the middle of the most notorious and pivotal semester, one that would determine my internship for the…
Extending the Context Window of LLaMA Models Paper Reading
Introduction During this week’s paper reading event, we are thrilled to announce that we will be joined by Frank Liu, Director of Operations, and ML Architect at Zilliz, who will be sharing valuable insights with us. This paper examines Position Interpolation (PI), a method extending context window sizes of LLaMA models up to 32,768 positions…
Llama 2: Open Foundation and Fine-Tuned Chat Models Paper Reading
Introduction In this paper reading, we explore the paper “Llama 2: Open Foundation and Fine-Tuned Chat Models.” The paper introduces Llama 2, a collection of pretrained and fine-tuned large language models ranging from 7 billion to 70 billion parameters. Their fine-tuned model, Llama 2-Chat, is specifically designed for dialogue use cases and showcases superior performance…
Modelbit + Arize: Enabling Rapid ML Model Deployment and Monitoring
This is a guest post authored by Michael Butler from Modelbit Seemingly every day a new open source model is announced with the potential to outperform any of the models that ML teams have already spent months setting up in production. Websites like Hugging Face have made it easy to pull a new model down…
Lost in the Middle: How LLMs Use Long Contexts
Introduction This paper examines how well language models utilize longer input contexts. The study focuses on multi-document question answering and key-value retrieval tasks. The researchers find that performance is highest when relevant information is at the beginning or end of the context. Accessing information in the middle of long contexts leads to significant performance degradation….
