ARIZE GENERATIVE AI COURSE
Intro To LLMOps
Intro To LLMOps LLM Agents: How To Set Up
Can an agent resolve whether extraterrestrial life exists?
Follow along with the accompanying notebook.
The rapid adoption of large language models continues to astound many industry watchers. In all, over half of machine learning teams are planning production deployments of LLMs this year – and nearly one in ten (8.3%) are already in production!
AI agents, chains and tools are a key part of how companies across industries are realizing that vision.
Chains are a hardcoded sequence of actions, agents use LLMs to reason which sequence of actions to take, and tools are functions that an agent calls.
This code-along post is written to help you use agents, chains, and tools to provide more relevant and detailed information on any topic – even the search for extraterrestrial life!
You will tackle a research goal by:
- Creating a LangChain single shot agent to aid in your research
- Give this agent access to a variety of tools, including the internet
- Leverage LLM summarization, chunking and chatbot capabilities
Your first research goal will be to prove the existence of extraterrestrial life.
Getting Started
The first step is to install and import the required modules, LLMs, and language tools.
!pip -q install langchain huggingface_hub openai google-search-results tiktoken wikipediaNext, you will need to obtain API keys for both OpenAI and SerpAPI.
import os
os.environ["OPENAI_API_KEY"] = "INSERT-API-KEY-HERE" # https://platform.openai.com
os.environ['SERPAPI_API_KEY'] = 'INSERT-API-KEY-HERE' # https://serpapi.com/Creating and Initializing the AI Agent
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.llms import OpenAIFirst, let’s load the language model we’re going to use to control the agent. Temp = 0 generally means less hallucinations and a more deterministic model.
The tools the agent will have access to are: [Google search, math, wikipedia, terminal]
Tools are functions that an agent calls. There are two important considerations here:
- Giving the agent access to the right tools; and
- Describing the tools in a way that is most helpful to the agent.
Without both, the agent you are trying to build will not work. If you don’t give the agent access to a correct set of tools, it will never be able to accomplish the objective. Similarly, describing the tools properly is important for the agent to know how to properly use them.
agent = initialize_agent(tools,
llm,
agent="zero-shot-react-description",
verbose=True)If you ever need to look up a tool, this is useful:
tools[1].name, tools[1].description
('Calculator', 'Useful for when you need to answer questions about math.')Chains
Using an LLM in isolation is fine for simple applications, but more complex applications require chaining LLMs — either with each other or with other components.
LangChain provides the chain interface for such “chained” applications. We define a chain very generically as a sequence of calls to components, which can include other chains.
Chains allow us to combine multiple components together to create a single, coherent application. For example, we can create a chain that takes user input, formats it with a PromptTemplate, and then passes the formatted response to an LLM. We can build more complex chains by combining multiple chains together, or by combining chains with other components.
agent.agent.llm_chain.prompt.template| Answer the following questions as best you can. You have access to the following tools:Search: A search engine. Useful for when you need to answer questions about current events. Input should be a search query. Calculator: Useful for when you need to answer questions about math. Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, facts, historical events, or other subjects. Input should be a search query. terminal: Run shell commands on this Linux machine. Use the following format: Question: the input question you must answer Begin! Question: {input} |
The core idea of agents is to use an LLM to choose a sequence of actions to take. In chains, a sequence of actions is hardcoded. In agents, a language model is used as a reasoning engine to determine which actions to take and in which order.
To get started, let’s run our agent with a few straightforward questions. Aliens are a popular topic of discussion, so there must be a lot of information available.
agent.run("What do we currently know about aliens?")| > Entering new AgentExecutor chain… Action: Search Action Input: “aliens” Observation: After floating in space for 57 years, Lt. Ripley’s (Sigourney Weaver) shuttle is found by a deep space salvage team. Upon arriving at LV-426, the marines find only one survivor, a nine year old girl named Newt (Carrie Henn). But even these battle-hardened marines with all the latest weaponry are no match for the hundreds of aliens that have invaded the colony.… MORE Thought: This is a movie description, not the answer I’m looking for. Action: Search Action Input: “aliens facts” Observation: Extraterrestrial life or alien life is life which may occur outside Earth and which did not originate on Earth. No extraterrestrial life has yet been conclusively detected. Thought: This is the answer I’m looking for. Final Answer: Extraterrestrial life or alien life is life which may occur outside Earth and which did not originate on Earth. No extraterrestrial life has yet been conclusively detected.> Finished chain. Extraterrestrial life or alien life is life which may occur outside Earth and which did not originate on Earth. No extraterrestrial life has yet been conclusively detected. |
Next, we ask about leading experts.
agent_output_UFO_expert = agent.run("Who is the leading expert on UFOs?")| > Entering new AgentExecutor chain… I should look for someone who has written extensively on the subject. Action: Search Action Input: “leading expert on UFOs” Observation: 1939) computer scientist and author, important figure in the UFO studies in France and in the United States. Promoted the extraterrestrial hypothesis and later … Thought: This looks like a promising lead. Action: Wikipedia Action Input: Jacques Vallee Observation: Page: Jacques Vallée Summary: Jacques Fabrice Vallée (French: [ʒak fabʁis vale]; born September 24, 1939) is an Internet pioneer, computer scientist, venture capitalist, author, ufologist and astronomer currently residing in San Francisco, California and Paris, France. His scientific career began as a professional astronomer at the Paris Observatory. Vallée co-developed the first computerized map of Mars for NASA in 1963. He later worked on the network information center for the ARPANET, a precursor to the modern Internet, as a staff engineer of SRI International’s Augmentation Research Center under Douglas Engelbart. Vallée is also an important figure in the study of unidentified flying objects (UFOs), first noted for a defense of the scientific legitimacy of the extraterrestrial hypothesis and later for promoting the interdimensional hypothesis.Page: Interdimensional UFO hypothesis Summary: The interdimensional hypothesis is a proposal that unidentified flying object (UFO) sightings are the result of experiencing other “dimensions” that coexist separately alongside our own in contrast with either the extraterrestrial hypothesis that suggests UFO sightings are caused by visitations from outside the Earth or the psychosocial hypothesis that argues UFO sightings are best explained as psychological or social phenomenon. The hypothesis has been advanced by ufologists such as Meade Layne, John Keel, J. Allen Hynek, and Jacques Vallée. Proponents of the interdimensional hypothesis argue that UFOs are a modern manifestation of a phenomenon that has occurred throughout recorded human history, which in prior ages were ascribed to mythological or supernatural creatures.Jeffrey J. Kripal, Chair in Philosophy and Religious Thought at Rice University, writes: “this interdimensional reading, long a staple of Spiritualism through the famous ‘fourth dimension’, would have a very long life within ufology and is still very much with us today”. Page: Dropa stones > Finished chain. |
Finally, we dive deeper into one of these.
agent.run("Take the year" + agent_output_UFO_expert + "died and add 100 years. Will this be the year we prove aliens exist?")| > Entering new AgentExecutor chain… I need to calculate the year Jacques Vallee died and add 100 years to it. Action: Calculator Action Input: 2019 + 100 Observation: Answer: 2119 Thought: I now know the year we will prove aliens exist, if we do. Final Answer: No, 2119 is not the year we will prove aliens exist.> Finished chain. No, 2119 is not the year we will prove aliens exist. |
💡Remember: Tools are functions that agents can use to interact with the world. These tools can be generic utilities (e.g. search), other chains, or even other agents. Note how our single agent is deciding between using search, math, or Wikipedia to deliver the best response.
agent.run("Is" + agent_output_UFO_expert + "dead?")| > Entering new AgentExecutor chain… I should find out if Jacques Vallee is still alive. Action: Wikipedia Action Input: Jacques Vallee Observation: Page: Jacques Vallée Summary: Jacques Fabrice Vallée (French: [ʒak fabʁis vale]; born September 24, 1939) is an Internet pioneer, computer scientist, venture capitalist, author, ufologist and astronomer currently residing in San Francisco, California and Paris, France. His scientific career began as a professional astronomer at the Paris Observatory. Vallée co-developed the first computerized map of Mars for NASA in 1963. He later worked on the network information center for the ARPANET, a precursor to the modern Internet, as a staff engineer of SRI International’s Augmentation Research Center under Douglas Engelbart. Vallée is also an important figure in the study of unidentified flying objects (UFOs), first noted for a defense of the scientific legitimacy of the extraterrestrial hypothesis and later for promoting the interdimensional hypothesis.Page: Interdimensional UFO hypothesis Summary: The interdimensional hypothesis is a proposal that unidentified flying object (UFO) sightings are the result of experiencing other “dimensions” that coexist separately alongside our own in contrast with either the extraterrestrial hypothesis that suggests UFO sightings are caused by visitations from outside the Earth or the psychosocial hypothesis that argues UFO sightings are best explained as psychological or social phenomenon. The hypothesis has been advanced by ufologists such as Meade Layne, John Keel, J. Allen Hynek, and Jacques Vallée. Proponents of the interdimensional hypothesis argue that UFOs are a modern manifestation of a phenomenon that has occurred throughout recorded human history, which in prior ages were ascribed to mythological or supernatural creatures.Jeffrey J. Kripal, Chair in Philosophy and Religious Thought at Rice University, writes: “this interdimensional reading, long a staple of Spiritualism through the famous ‘fourth dimension’, would have a very long life within ufology and is still very much with us today”. Page: Dropa stones > Finished chain. Jacques Vallee is still alive and is a leading expert on UFOs, but the Dropa stones are likely a hoax. |
Refining
The use of a single agent with tools is useful, but is starting to hallucinate. If we want correct and reliable information, maybe we should try to look at alternative words for UFOs and aliens…
#ask questions that can help you search for the most relevant research
agent.run("What are alternative names for UFOs?")| > Entering new AgentExecutor chain… I should look for information about UFOs. Action: Wikipedia Action Input: UFOPage: Unidentified flying object Summary: An unidentified flying object (UFO), or unidentified aerial (or anomalous) phenomenon (UAP), is any perceived aerial phenomenon that cannot be immediately identified or explained. Upon investigation, most UFOs are identified as known objects or atmospheric phenomena, while a small number remain unexplained. While unusual sightings have been reported in the sky throughout history, UFOs became culturally prominent after World War II, escalating during the Space Age. Studies and investigations into UFO reports conducted by governments (such as Project Blue Book in the United States and Project Condign in the United Kingdom), as well as by organisations and individuals have occurred over the years without confirmation of the fantastical claims of believers. The U.S. government currently has two entities dedicated to UAP (or UFO) data collection and analysis: NASA’s UAP independent study team and the All-domain Anomaly Resolution Office. Scientists and skeptic organizations such as the Committee for Skeptical Inquiry have provided prosaic explanations for UFOs, namely that they are caused by natural phenomena, human technology, delusions, and hoaxes. Small but vocal groups of ufologists favour unconventional or pseudoscientific hypotheses, often claiming that UFOs are evidence of extraterrestrial intelligence, technologically advanced cryptids or interdimensional contact, but even after decades of promotion of such ideas by believers and in popular media, the kind of evidence required to solidly support such claims has not been forthcoming. Beliefs surrounding UFOs have inspired parts of new religions even as social scientists have identified the ongoing interest and storytelling surrounding UFOs as a modern example of folklore and mythology understandable with psychosocial explanations. Page: Roswell incident > Finished chain. Alternative names for UFOs include Unidentified Flying Object (UFO), Unidentified Aerial Phenomenon (UAP), and Flying Disc. |
You can also ask multiple questions at once.
gent.run("How are Foo Fighters a name for a UFO? And is Dave Grohl human?")| > Entering new AgentExecutor chain… I should research the Foo Fighters and Dave Grohl Action: Search Action Input: “Foo Fighters” “Dave Grohl” Observation: Foo Fighters is an American rock band formed in Seattle in 1994. Foo Fighters was initially formed as a one-man project by former Nirvana drummer Dave Grohl. Following the success of the 1995 eponymous debut album, Grohl recruited a band consisting of Nate Mendel, William Goldsmith, and Pat Smear. Thought: I should research if Dave Grohl is human Action: Search Action Input: “Dave Grohl human” Observation: They’re human beings. And there has to be some sort of compassion or understanding that their lives should be their lives. That it’s not all … Thought: I now know the final answer Final Answer: Dave Grohl is a human being and the name Foo Fighters is derived from the term used by Allied aircraft pilots in World War II to describe UFOs.> Finished chain. Dave Grohl is a human being and the name Foo Fighters is derived from the term used by Allied aircraft pilots in World War II to describe UFOs. |
Good to know Dave Grohl was human (I wasn’t sure!). It is also good to know a UFO is now being referred to as a UAP – worth keeping in mind for future research.
Now, let’s look up some outside resources to get answers about UFOs and UAPs…
agent.run("Are there annual UAP reports?")| > Entering new AgentExecutor chain… I should research this to find out. Action: Search Action Input: “annual UAP reports” Observation: The National Defense Authorization Act for Fiscal Year 2022 required the Director of National Intelligence, in consultation with the … Thought: This looks like the answer I’m looking for. Action: Wikipedia Action Input: “Unidentified Flying Object” Observation: Page: Unidentified flying object Summary: An unidentified flying object (UFO), or unidentified aerial (or anomalous) phenomenon (UAP), is any perceived aerial phenomenon that cannot be immediately identified or explained. Upon investigation, most UFOs are identified as known objects or atmospheric phenomena, while a small number remain unexplained. While unusual sightings have been reported in the sky throughout history, UFOs became culturally prominent after World War II, escalating during the Space Age. Studies and investigations into UFO reports conducted by governments (such as Project Blue Book in the United States and Project Condign in the United Kingdom), as well as by organisations and individuals have occurred over the years without confirmation of the fantastical claims of believers. The U.S. government currently has two entities dedicated to UAP (or UFO) data collection and analysis: NASA’s UAP independent study team and the All-domain Anomaly Resolution Office. Scientists and skeptic organizations such as the Committee for Skeptical Inquiry have provided prosaic explanations for UFOs, namely that they are caused by natural phenomena, human technology, delusions, and hoaxes. Small but vocal groups of ufologists favour unconventional or pseudoscientific hypotheses, often claiming that UFOs are evidence of extraterrestrial intelligence, technologically advanced cryptids or interdimensional contact, but even after decades of promotion of such ideas by believers and in popular media, the kind of evidence required to solidly support such claims has not been forthcoming. Beliefs surrounding UFOs have inspired parts of new religions even as social scientists have identified the ongoing interest and storytelling surrounding UFOs as a modern example of folklore and mythology understandable with psychosocial explanations. Page: Varginha UFO incident Page: Unidentified submerged object > Finished chain. Yes, there are annual UAP reports, which are collected and analyzed by the U.S. government. Additionally, there have been reports of USOs, which are unidentified submerged objects. |
Interesting! Diving deeper:
agent.run("Where can I download the '2022 ANNUAL REPORT ON UNIDENTIFIED AERIAL PHENOMENA'? What is the url?")| > Entering new AgentExecutor chain… I need to find a website that has the report. Action: Search Action Input: “2022 ANNUAL REPORT ON UNIDENTIFIED AERIAL PHENOMENA” Observation: The National Defense Authorization Act for Fiscal Year 2022 required the Director of National Intelligence, in consultation with the … Thought: This looks like a good source. Action: Wikipedia Action Input: “National Defense Authorization Act for Fiscal Year 2022” Observation: Page: National Defense Authorization Act for Fiscal Year 2022 Summary: The National Defense Authorization Act for Fiscal Year 2022 (S. 1605; NDAA 2022, Pub.L. 117-81) is a United States federal law which specifies the budget, expenditures and policies of the U.S. Department of Defense (DOD) for fiscal year 2022. Analogous NDAAs have been passed annually for 60 years. Page: National Defense Authorization Act Page: National Defense Authorization Act for Fiscal Year 2023 > Finished chain. The 2022 ANNUAL REPORT ON UNIDENTIFIED AERIAL PHENOMENA can be downloaded from the website of the U.S. Department of Defense: https://www.defense.gov/Newsroom/Releases/NDAA-2022/ |
Diving still deeper:
agent.run("Is there a pdf file for that report?")| > Entering new AgentExecutor chain… I need to find out if the report is available in pdf format. Action: Search Action Input: “report name” pdf Observation: Report Name Paper. pdfFiller is the best quality online PDF editor and form builder – it’s fast, secure and easy to use. Edit, sign, fax and print documents … Thought: I now know the final answer Final Answer: Yes, the report is available in pdf format.> Finished chain. Yes, the report is available in pdf format. |
Now that we know there is a full report available, Iet’s going to download it.
However, before we read through the entire report, let’s summarize it and then train a chatbot on it to make sure it contains the information needed.
pip install "unstructured[all-docs]" #let's save the pdf via URLfrom langchain.document_loaders import UnstructuredURLLoaderurls = ["https://www.dni.gov/files/ODNI/documents/assessments/Unclassified-2022-Annual-Report-UAP.pdf"] #location of the reportloader = UnstructuredURLLoader(urls=urls)
TOP_SECRET_FILE = loader.load() #clearly I need to save this as a secret fileTime to summarize the report:
from langchain.document_loaders import UnstructuredFileLoader
from langchain.chains.summarize import load_summarize_chain
from langchain.chains.question_answering import load_qa_chain
def doc_summary(docs):
print (f'You have {len(docs)} document(s)')
num_words = sum([len(doc.page_content.split(' ')) for doc in docs])
print (f'You have roughly {num_words} words in your docs')
print ()
print (f'Preview: \n{docs[0].page_content.split(". ")[0]}')doc_summary(TOP_SECRET_FILE)Unfortunately, we get the following error:
| InvalidRequestError Traceback (most recent call last)in () 1 chain = load_summarize_chain(llm, chain_type=”map_reduce”, verbose=True) —-> 2 chain.run(TOP_SECRET_FILE) #This will only work if we do not exceed the token limit 25 frames InvalidRequestError: This model’s maximum context length is 4097 tokens, however you requested 5156 tokens (4900 in your prompt; 256 for the completion). Please reduce your prompt; or completion length. |
Our document is too long if we load it all at once! So we will need to chunk the file with LangChain’s text splitter.
from langchain.text_splitter import RecursiveCharacterTextSplittertext_splitter = RecursiveCharacterTextSplitter(
# Set a really small chunk size, just to show.
chunk_size = 600,
chunk_overlap = 0
)
TOP_SECRET_FILE = text_splitter.split_documents(TOP_SECRET_FILE)
doc_summary(TOP_SECRET_FILE)Chain.run(TOP_SECRET_FILE[1:40])Once this process is complete, we get the following:
| > Finished chain.This article outlines the governmental changes to manage Unidentified Aerial Phenomena (UAP) issues, including the establishment of the All-Domain Anomaly Resolution Office (AARO) and the Office of the Director of National Intelligence (ODNI) coordinating with AARO and multi-agency partners to assist in the collection, reporting, and analysis of UAP incidents across the Intelligence Community. AARO is leveraging an analytic process to analyze UAP reports, and the ODNI and AARO are committed to responsibly sharing findings with aviators and operators. The Department of Defense and Intelligence Community are working together to report, identify, and resolve UAP events, and the FY 2022 National Defense Authorization Act (NDAA) established the Unidentified Aerial Phenomena (UAP) Task Force, now known as the Aerospace Threat Identification Program (AARO). AARO is working with the IC to manage the influx of data from increased UAP events, including intake, indexing, visualization, and analysis. |
Great! This report seems to contain real information on UFOs.
Let’s say that we don’t really want to read the entire report to see if it can answer questions for an article I’m writing, but are mainly interested in whether this report tells us whether we are in danger (from what we currently know about UFOs) and if the report provides evidence that aliens exist.
Before reading the full report, let’s just ask the report if it can answer these questions.
Q&A Map Re-Rank
The map re-rank documents chain runs an initial prompt on each document that not only tries to complete a task but also gives a score for how certain it is in its answer. The highest scoring response is returned.
chain = load_qa_chain(llm, chain_type="map_rerank", verbose=True, return_intermediate_steps=True)
#ask the report your important questions
query = "Are UAPs a danger to our safety? Does this mean that aliens are real?"
result = chain({"input_documents": TOP_SECRET_FILE[1:40], "question": query}, return_only_outputs=True)Next, let’s get the result:
result['output_text']| UAPs have not been proven to be a danger to our safety, and this document does not provide evidence that aliens are real. |
One out of two answers (not bad). Let’s check intermediate steps first just to check that these answers are reliable, as well as where exactly in the document chunks the answers were found.
result['intermediate_steps']| [{‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ UAPs have not been proven to be a danger to our safety, and this document does not provide evidence that aliens are real.’, ‘score’: ’80’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ UAPs may represent a safety of flight hazard, but this document does not answer the question of whether aliens are real.’, ‘score’: ’40’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ UAPs may pose a danger to our safety, but this document does not provide evidence of alien involvement.’, ‘score’: ’60’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ UAPs pose a potential risk to flight safety, but there is no evidence to suggest that aliens are real.’, ‘score’: ’80’}, {‘answer’: ‘ Yes, UAPs pose a safety of flight and collision hazard to air assets. No, this does not mean that aliens are real.’, ‘score’: ’80’}, {‘answer’: ‘ There have been no reported collisions between U.S. aircraft and UAP, and no encounters with UAP confirmed to contribute directly to adverse health-related effects to the observer(s).’, ‘score’: ’80’}, {‘answer’: ‘ UAPs represent a hazard to flight safety and pose a possible adversary. This does not necessarily mean that aliens are real.’, ‘score’: ’80’}, {‘answer’: ‘ UAPs pose a potential risk to safety of flight and could be adversarial activity. There is no evidence to suggest that aliens are real.’, ‘score’: ’80’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}, {‘answer’: ‘ This document does not answer the question.’, ‘score’: ‘0’}] |
Since reports and searching the web are not yielding the answers we need about aliens, let’s see if eye-witness testimonies from YouTube will help!
YouTube Exercise
We are now going to pull videos from YouTube and summarize their findings. We can achieve this with the OpenAIWhisperParser, which will use the OpenAI Whisper API to transcribe audio to text, and the OpenAIWhisperParserLocal for local support and running on private clouds or on premise.
from langchain.document_loaders.generic import GenericLoader
from langchain.document_loaders.blob_loaders.youtube_audio import YoutubeAudioLoaderfrom langchain.document_loaders.parsers import OpenAIWhisperParserLet’s use yt_dlp to download audio for YouTube urls. We will use pydub to split downloaded audio files (such that we adhere to Whisper API’s 25MB file size limit).
pip install yt_dlp
pip install pydub
pip install librosaWe will need to go from YouTube URL to text.
pip install youtube_searchfrom langchain.tools import YouTubeSearchToolAgent_youtube = YouTubeSearchTool()Agent_youtube.run("UFO breaking news,3") #search YouTube for the top 3 breaking news videos on UFOs| [‘/watch?v=15FoQ37js_8&pp=ygURVUZPIGJyZWFraW5nIG5ld3M%3D’, ‘/watch?v=470K8IZrLRE&pp=ygURVUZPIGJyZWFraW5nIG5ld3M%3D’, ‘/watch?v=TjzHL1hVo2o&pp=ygURVUZPIGJyZWFraW5nIG5ld3M%3D’] |
In order to visit YouTube and check out these videos, you can enter the URLs directly into the search window by adding https://www.youtube.com before the results.
The agent recommends three videos. After a quick check, it is apparent that all three videos are from reputable sources, are the latest videos (in fact the first video as of this writing is currently running live from CBS news) and are all extremely relevant.
# Let's save the two videos which aren't currently live!
urls = ["https://www.youtube.com/watch?v=15FoQ37js_8&t=4s", "https://www.youtube.com/watch?v=TjzHL1hVo2o&t=1s"]We will use the YoutubeAudioLoader to fetch and download the audio files.
Then, use OpenAIWhisperParser() to transcribe them to text.
agent.run("what current directory am I in?") #Yes, your agent knows Linux| > Entering new AgentExecutor chain… I need to use a terminal command to find out Action: terminal Action Input: pwd Observation: /contentThought: I now know the final answer Final Answer: The current directory is /content. > Finished chain. The current directory is /content. |
# Directory to save audio files
save_dir = "/content/YouTube"# set a flag to switch between local and remote parsing
# change this to True if you want to use local parsing
local = FalseNext, let’s transcribe the videos to text:
if local:
loader = GenericLoader(YoutubeAudioLoader(urls, save_dir), OpenAIWhisperParserLocal())
else:
loader = GenericLoader(YoutubeAudioLoader(urls, save_dir), OpenAIWhisperParser())
docs = loader.load()
# Returns a list of Documents, which can be easily viewed or parsed
docs[0].page_content[0:500]Building a Chatbot from the YouTube Videos
Similarly to how we build a chat bot on our TOP SECRET REPORT, now that we have downloaded, transcribed and saved the most recent and reliable YouTube videos on UAPs, we can also create a Q&A system for them.
First, let’s install Faiss. Faiss is a library for efficient similarity search and clustering of dense vectors and it’s an indexing method for vectorDBs.
pip install faiss-cpu google-search-resultsfrom langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings
from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAINext, let’s combine the docs:
combined_docs = [doc.page_content for doc in docs]
text = " ".join(combined_docs)Split them:
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1500, chunk_overlap=150)
splits = text_splitter.split_text(text)embeddings = OpenAIEmbeddings()
vectordb = FAISS.from_texts(splits, embeddings)Build a QA chain:
qa_chain = RetrievalQA.from_chain_type(
llm=ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0),
chain_type="stuff",
retriever=vectordb.as_retriever(),
)Then, we get to the fun stuff:
query = "Do we have any UAP sightings?"
qa_chain.run(query)| Yes, there have been multiple UAP sightings reported by credible sources, including fighter pilots and naval aviators. These sightings describe objects with flight characteristics that surpass our most advanced fighter jets. |
query = "Is there any evidence of UAP crash retrival?"
qa_chain.run(query)| Based on the given context, there is no direct evidence mentioned regarding UAP crash retrieval. The discussion revolves around two potential conclusions: either there has been a decades-long cover-up of UFO retrieval and reverse engineering activities, or there is a psychological disinformation campaign. However, no specific evidence or incidents of UAP crash retrieval are mentioned in this conversation. |
query = "Have aliens, UFOs or any non-human body been recovered at crash sites for UAPS?"
qa_chain.run(query)| According to the provided context, there are claims made by witnesses and experts that suggest the existence of evidence such as recovered crafts and possibly even bodies related to UFO encounters. However, without further concrete evidence or official confirmation, it is not possible to definitively answer whether aliens, UFOs, or non-human bodies have been recovered at crash sites for UAPs. |
query = "Do aliens exist?"
qa_chain.run(query)| I don’t know if aliens exist. There is ongoing speculation and investigation into unidentified flying objects (UFOs) and the possibility of extraterrestrial life, but conclusive evidence has not been presented. |
query = "Is there a UFO cover-up?"
qa_chain.run(query)| According to the context provided, there are two possible conclusions: either the United States government has been covering up UFO retrieval and reverse engineering activities for decades, or there is a psychological disinformation campaign being conducted by elements of the defense and intelligence establishment. However, it is important to note that this is just one perspective and there is no definitive evidence to confirm or deny a UFO cover-up. |
To summarize, from these YouTube videos we know that there have been multiple UAP sightings reported by credible sources, including fighter pilots and naval aviators. While we don’t have conclusive evidence that aliens exist, there is an ongoing investigation around whether the United States government has been covering up UFO retrieval and reverse engineering activities for decades. Experts also claim the existence of evidence such as recovered crafts and potentially even bodies related to UFO encounters.
Further Analysis
Let’s keep diving deeper.
agent.run("What is the Drake equation?")| > Entering new AgentExecutor chain… I should look up the Drake equation on Wikipedia. Action: Wikipedia Action Input: Drake equation /usr/local/lib/python3.10/dist-packages/wikipedia/wikipedia.py:389: GuessedAtParserWarning: No parser was explicitly specified, so I’m using the best available HTML parser for this system (“lxml”). This usually isn’t a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.The code that caused this warning is on line 389 of the file /usr/local/lib/python3.10/dist-packages/wikipedia/wikipedia.py. To get rid of this warning, pass the additional argument ‘features=”lxml”‘ to the BeautifulSoup constructor. lis = BeautifulSoup(html).find_all(‘li’) Observation: Page: Drake equation Page: Frank Drake > Finished chain. The Drake equation is a probabilistic argument used to estimate the number of active, communicative extraterrestrial civilizations in the Milky Way Galaxy. It was formulated in 1961 by Frank Drake, not for purposes of quantifying the number of civilizations, but as a way to stimulate scientific dialogue at the first scientific meeting on the search for extraterrestrial intelligence (SETI). The equation summarizes the main concepts which scientists must contemplate when considering the question of other radio-communicative life. Criticism related to the Drake equation focuses not on the equation itself, but on the fact that the estimated values for several of its factors are highly conjectural, the combined multiplicative effect being that the uncertainty associated with any derived value is so large that the equation cannot be used to draw firm conclusions. |
From there, crunching the numbers is straightforward:
agent.run("Take the number habitable planets are in our galaxy and multiply it by 7 billion")| > Entering new AgentExecutor chain… I need to find the number of habitable planets in our galaxy and then multiply it by 7 billion Action: Search Action Input: Number of habitable planets in our galaxy Observation: So far, researchers have identified several hundred planets in the habitable zone of their star in Kepler data. It may take a while to find all … Thought: I need to find a more exact number Action: Wikipedia Action Input: Habitable zone Observation: Page: Circumstellar habitable zone Summary: In astronomy and astrobiology, the circumstellar habitable zone (CHZ), or simply the habitable zone, is the range of orbits around a star within which a planetary surface can support liquid water given sufficient atmospheric pressure. The bounds of the CHZ are based on Earth’s position in the Solar System and the amount of radiant energy it receives from the Sun. Due to the importance of liquid water to Earth’s biosphere, the nature of the CHZ and the objects within it may be instrumental in determining the scope and distribution of planets capable of supporting Earth-like extraterrestrial life and intelligence. The habitable zone is also called the Goldilocks zone, a metaphor, allusion and antonomasia of the children’s fairy tale of “Goldilocks and the Three Bears”, in which a little girl chooses from sets of three items, ignoring the ones that are too extreme (large or small, hot or cold, etc.), and settling on the one in the middle, which is “just right”. Since the concept was first presented in 1953, many stars have been confirmed to possess a CHZ planet, including some systems that consist of multiple CHZ planets. Most such planets, being either super-Earths or gas giants, are more massive than Earth, because massive planets are easier to detect. On November 4, 2013, astronomers reported, based on Kepler data, that there could be as many as 40 billion Earth-sized planets orbiting in the habitable zones of Sun-like stars and red dwarfs in the Milky Way. About 11 billion of these may be orbiting Sun-like stars. Proxima Centauri b, located about 4.2 light-years (1.3 parsecs) from Earth in the constellation of Centaurus, is the nearest known exoplanet, and is orbiting in the habitable zone of its star. The CHZ is also of particular interest to the emerging field of habitability of natural satellites, because planetary-mass moons in the CHZ might outnumber planets.In subsequent decades, the CHZ concept began to be challenged as a primary criterion for life, so the concept is still evolving. Since the discovery of evidence for extraterrestrial liquid water, substantial quantities of it are now thought to occur outside the circumstellar habitable zone. The concept of deep biospheres, like Earth’s, that exist independently of stellar energy, are now generally accepted in astrobiology given the large amount of liquid water known to exist within in lithospheres and asthenospheres of the Solar System. Sustained by other energy sources, such as tidal heating or radioactive decay or pressurized by non-atmospheric means, liquid water may be found even on rogue planets, or their moons. Liquid water can also exist at a wider range of temperatures and pressures as a solution, for example with sodium chlorides in seawater on Earth, chlorides and sulphates on equatorial Mars, or ammoniates, due to its different colligative properties. In addition, other circumstellar zones, where non-water solvents favorable to hypothetical life based on alternative biochemistries could exist in liquid form at the surface, have been proposed. Page: Galactic habitable zone > Finished chain. 2800000000000 |
Finally, an important question:
agent.run("Has the US Government ever covered anything up?")| > Entering new AgentExecutor chain… I should research this question to find out. Action: Wikipedia Action Input: “US Government cover-ups” Observation: Page: UPS Airlines Flight 6 Summary: UPS Airlines Flight 6 was a cargo flight operated by UPS Airlines. On September 3, 2010, the Boeing 747-400F flying the route between Dubai, United Arab Emirates, and Cologne, Germany, developed an in-flight fire, which caused the aircraft to crash, killing both crew members, the only people on board. It was the first fatal air crash for UPS Airlines. The crash prompted a re-evaluation of safety procedures protecting airliners from cockpit smoke. Page: American cover-up of Japanese war crimes Page: UPS Airlines > Finished chain. Yes, the US Government has covered up things in the past. Examples include the pardon of Japanese war criminals after World War II and the cover-up of the UPS Airlines Flight 6 crash in 2010. |
Conclusion
Of course, the search for extraterrestrial life is just one fun example. Here are a few things to keep in mind as you begin to experiment:
- Agents, chains, and tools are all individually useful, but they provide the most relevant information when leveraged together.
- When you give your agent and LLM chains access to the internet, the quality of the retrieved query is greatly improved. More work is needed to quantitatively evaluate agents’ performance, but you can start by reading the intermediate steps.
- Regardless of your query, the truth is out there!
