Chains and Why They Are Used

Introduction

Prompting is considered the most effective method of interacting with language models as it enables querying information using natural language. We already went through the prompting techniques and briefly used chains earlier. In this lesson, the chains will explain the chains in more detail.

The chains are responsible for creating an end-to-end pipeline for using the language models. They will join the model, prompt, memory, parsing output, and debugging capability and provide an easy-to-use interface. A chain will 1) receive the user’s query as an input, 2) process the LLM’s response, and lastly, 3) return the output to the user.

It is possible to design a custom pipeline by inheriting the Chain class. For example, the LLMChain is the simplest form of chain in LangChain, inheriting from the Chain parent class. We will start by going through ways to invoke this class and follow it by looking at adding different functionalities.

LLMChain

Several methods are available for utilizing a chain, each yielding a distinct output format. The example in this section is creating a bot that can suggest a replacement word based on context. The code snippet below demonstrates the utilization of the GPT-3 model through the OpenAI API. It generates a prompt using the PromptTemplate from LangChain, and finally, the LLMChain class ties all the components. Also, It is important to set the OPENAI_API_KEY environment variable with your API credentials from OpenAI. Remember to install the required packages with the following command: pip install langchain==0.1.4 deeplake openai==1.10.0 tiktoken.

from langchain import PromptTemplate, OpenAI, LLMChain

prompt_template = "What is a word to replace the following: {word}?"

# Set the "OPENAI_API_KEY" environment variable before running following line.
llm = OpenAI(model_name="gpt-3.5-turbo-instruct", temperature=0)

llm_chain = LLMChain(
    llm=llm,
    prompt=PromptTemplate.from_template(prompt_template)
)

The most straightforward approach uses the chain class __call__ method. It means passing the input directly to the object while initializing it. It will return the input variable and the model’s response under the text key.

llm_chain("artificial")

{'word': 'artificial', 'text': '\n\nSynthetic'}

It is also possible to use the .apply() method to pass multiple inputs at once and receive a list for each input. The sole difference lies in the exclusion of inputs within the returned list. Nonetheless, the returned list will maintain the identical order as the input.

input_list = [
    {"word": "artificial"},
    {"word": "intelligence"},
    {"word": "robot"}
]

llm_chain.apply(input_list)

[{'text': '\n\nSynthetic'}, {'text': '\n\nWisdom'}, {'text': '\n\nAutomaton'}]

The .generate() method will return an instance of LLMResult, which provides more information. For example, the finish_reason key indicates the reason behind the stop of the generation process. It could be stopped, meaning the model decided to finish or reach the length limit. There is other self-explanatory information like the number of total used tokens or the used model.

llm_chain.generate(input_list)

LLMResult(generations=[[Generation(text='\n\nSynthetic', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text='\n\nWisdom', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text='\n\nAutomaton', generation_info={'finish_reason': 'stop', 'logprobs': None})]], llm_output={'token_usage': {'prompt_tokens': 33, 'completion_tokens': 13, 'total_tokens': 46}, 'model_name': 'gpt-3.5-turbo-instruct'})

The next method we will discuss is .predict(). (which could be used interchangeably with .run()) Its best use case is to pass multiple inputs for a single prompt. However, it is possible to use it with one input variable as well. The following prompt will pass both the word we want a substitute for and the context the model must consider.

prompt_template = "Looking at the context of '{context}'. What is an appropriate word to replace the following: {word}?"

llm_chain = LLMChain(
    llm=llm,
    prompt=PromptTemplate(template=prompt_template, input_variables=["word", "context"]))

llm_chain.predict(word="fan", context="object")
# or llm_chain.run(word="fan", context="object")

'\n\nVentilator'

The model correctly suggested that a Ventilator would be a suitable replacement for the word fan in the context of objects. Furthermore, when we repeat the experiment with a different context, humans, the output will change the Admirer.

llm_chain.predict(word="fan", context="humans")
# or llm_chain.run(word="fan", context="humans")

'\n\nAdmirer'

The sample codes above show how passing single or multiple inputs to a chain and retrieving the outputs is possible. However, we prefer to receive a formatted output in most cases, as we learned in the “Managing Outputs with Output Parsers” lesson.

Parsers

As discussed, the output parsers can define a data schema to generate correctly formatted responses. It wouldn’t be an end-to-end pipeline without using parsers to extract information from the LLM textual output. The following example shows the use of CommaSeparatedListOutputParser class with the PromptTemplate to ensure the results will be in a list format.

from langchain.output_parsers import CommaSeparatedListOutputParser

output_parser = CommaSeparatedListOutputParser()
template = """List all possible words as substitute for 'artificial' as comma separated."""

llm_chain = LLMChain(
    llm=llm,
    prompt=PromptTemplate(template=template, output_parser=output_parser, input_variables=[]),
    output_parser=output_parser)

llm_chain.predict()

['Synthetic',
 'Manufactured',
 'Imitation',
 'Fabricated',
 'Fake',
 'Simulated',
 'Artificial Intelligence',
 'Automated',
 'Constructed',
 'Programmed',
 'Processed',
 'Mechanical',
 'Man-Made',
 'Lab-Created',
 'Artificial Neural Network.']

Conversational Chain (Memory)

Depending on the application, memory is the next component that will complete a chain. LangChain provides a ConversationalChain to track previous prompts and responses using the ConversationalBufferMemory class.

from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory

output_parser = CommaSeparatedListOutputParser()
conversation = ConversationChain(
    llm=llm,
    memory=ConversationBufferMemory()
)

conversation.predict(input="List all possible words as substitute for 'artificial' as comma separated.")

'Synthetic, robotic, manufactured, simulated, computerized, programmed, man-made, fabricated, contrived, and artificial.'

Now, we can ask it to return the following four replacement words. It uses the memory to find the next options.

conversation.predict(input="And the next 4?")

'Automated, cybernetic, mechanized, and engineered.'

Sequential Chain

Another helpful feature is using a sequential chain that concatenates multiple chains into one. The following code shows a sample usage.

# poet
poet_template: str = """You are an American poet, your job is to come up with\
poems based on a given theme.

Here is the theme you have been asked to generate a poem on:
{input}\
"""

poet_prompt_template: PromptTemplate = PromptTemplate(
    input_variables=["input"], template=poet_template)

# creating the poet chain
poet_chain: LLMChain = LLMChain(
    llm=llm, output_key="poem", prompt=poet_prompt_template)

# critic
critic_template: str = """You are a critic of poems, you are tasked\
to inspect the themes of poems. Identify whether the poem includes romantic expressions or descriptions of nature.

Your response should be in the following format, as a Python Dictionary.
poem: this should be the poem you received 
Romantic_expressions: True or False
Nature_descriptions: True or False

Here is the poem submitted to you:
{poem}\
"""

critic_prompt_template: PromptTemplate = PromptTemplate(
    input_variables=["poem"], template=critic_template)

# creating the critic chain
critic_chain: LLMChain = LLMChain(
    llm=llm, output_key="critic_verified", prompt=critic_prompt_template)

In this example we define two processes in a chain: one for generating poems based on a given theme ("poet") and another for evaluating these poems on their romantic and natural elements ("critic"). The poet process uses a template to instruct an AI model to create poems, while the critic process analyzes these poems, flagging them for specific content. The setup utilizes prompt templates and chains to seamlessly integrate content generation with content verification.

from langchain.chains import SimpleSequentialChain

overall_chain = SimpleSequentialChain(chains=[poet_chain, critic_chain])

The SimpleSequentialChain will start running each chain from the first index and pass its response to the next one in the list.

# Run the poet and critic chain with a specific theme
theme: str = "the beauty of nature"
review = overall_chain.run(theme)

# Print the review to see the critic's evaluation
print(review)

The sample code.

poem: Nature's Beauty
Romantic_expressions: False
Nature_descriptions: True

The output.

Debug

It is possible to trace the inner workings of any chain by setting the verbose argument to True. As you can see in the following code, the chain will return the initial prompt and the output. The output depends on the application. It may contain more information if there are more steps.

template = """List all possible words as substitute for 'artificial' as comma separated.

Current conversation:
{history}

{input}"""

conversation = ConversationChain(
    llm=llm,
    prompt=PromptTemplate(template=template, input_variables=["history", "input"], output_parser=output_parser),
    memory=ConversationBufferMemory(),
    verbose=True)

conversation.predict(input="")

> Entering new ConversationChain chain...
Prompt after formatting:
List all possible words as substitute for 'artificial' as comma separated.

Current conversation:


Answer briefly. write the first 3 options.

> Finished chain.
'Synthetic, Imitation, Manufactured, Fabricated, Simulated, Fake, Artificial, Constructed, Computerized, Programmed'

Custom Chain

The LangChain library has several predefined chains for different applications like Transformation Chain, LLMCheckerChain, LLMSummarizationCheckerChain, and OpenAPI Chain, which all share the same characteristics mentioned in previous sections. It is also possible to define your chain for any custom task. In this section, we will create a chain that returns a word's meaning and then suggests a replacement.

It starts by defining a class that inherits most of its functionalities from the Chain class. Then, the following three methods must be declared depending on the use case. The input_keys and output_keys methods let the model know what it should expect, and a _call method runs each chain and merges their outputs.

from langchain.chains import LLMChain
from langchain.chains.base import Chain

from typing import Dict, List


class ConcatenateChain(Chain):
    chain_1: LLMChain
    chain_2: LLMChain

    @property
    def input_keys(self) -> List[str]:
        # Union of the input keys of the two chains.
        all_input_vars = set(self.chain_1.input_keys).union(set(self.chain_2.input_keys))
        return list(all_input_vars)

    @property
    def output_keys(self) -> List[str]:
        return ['concat_output']

    def _call(self, inputs: Dict[str, str]) -> Dict[str, str]:
        output_1 = self.chain_1.run(inputs)
        output_2 = self.chain_2.run(inputs)
        return {'concat_output': output_1 + output_2}

Then, we will declare each chain individually using the LLMChain class. Lastly, we call our custom chain ConcatenateChain to merge the results of the chain_1 and chain_2.

prompt_1 = PromptTemplate(
    input_variables=["word"],
    template="What is the meaning of the following word '{word}'?",
)
chain_1 = LLMChain(llm=llm, prompt=prompt_1)

prompt_2 = PromptTemplate(
    input_variables=["word"],
    template="What is a word to replace the following: {word}?",
)
chain_2 = LLMChain(llm=llm, prompt=prompt_2)

concat_chain = ConcatenateChain(chain_1=chain_1, chain_2=chain_2)
concat_output = concat_chain.run("artificial")
print(f"Concatenated output:\n{concat_output}")

Concatenated output:


Artificial means something that is not natural or made by humans but rather created or produced by artificial means.

Synthetic

Conclusion

This lesson taught us about LangChain and its powerful feature, chains, which combine multiple components to create a coherent application. The lesson initially showed the usage of several predefined chains from the LangChain library. Then, we built up by adding more features like parsers, memory, and debugging. Lastly, the process of defining custom chains was explained.

In the next lesson, we will do a hands-on project summarizing Youtube videos.

Resources

Chains | 🦜️🔗 Langchain

Using an LLM in isolation is fine for simple applications,

python.langchain.com

You can find the code of this lesson in this online Notebook.