​
Integrating Webpage Content into Your Search Agent

Adding individual webpages to your search agent allows for the inclusion of specific, high-quality content that can directly answer user queries. The add_webpage method streamlines the process of incorporating content from single webpages, enhancing the search agent’s ability to provide precise and relevant search results.

​
Function Signature

The add_webpage function is tailored for adding content from a single webpage, using a range of parameters to fine-tune how this content is processed and indexed.

​
Parameters

  • url (Optional[str]): The URL of the webpage to be added. This specifies the exact page whose content you wish to make searchable.
  • system_prompt (str): An optional prompt to guide the system in how to process the content of the webpage. It can be used to direct the focus of content extraction towards relevant information.
  • query_wrapper_prompt (str): An optional prompt that can enhance the relevance of search queries by wrapping them in a specific context related to the webpage content.
  • embed_model (Union[str, EmbedType]): The embedding model used for processing and embedding the webpage’s text content. Defaults to a standard model suitable for general web content.
  • llm_params (dict): Parameters for integrating Large Language Models to augment content understanding and query processing capabilities.
  • vector_store_params (dict): Configuration for the vector storage, specifying how and where the extracted content embeddings are stored.
  • service_context_params (dict): Additional parameters to customize the service context specific to the webpage’s content.
  • query_engine_params (dict): Customization parameters for the query engine, affecting how the webpage content is searched and matched with queries.
  • retriever_params (dict): Configuration for the document retriever component, determining how the webpage content is indexed and retrieved based on search queries.

​
Example Usage

​
Adding a Specific Webpage

search_agent.add_webpage(
    url="https://www.specificpage.com/article",
)

This example adds content from a specific webpage.