scrapegraphai.builders package¶

Submodules¶

scrapegraphai.builders.graph_builder module¶

GraphBuilder Module

class scrapegraphai.builders.graph_builder.GraphBuilder(prompt: str, config: dict)¶

Bases: object

GraphBuilder is a dynamic tool for constructing web scraping graphs based on user prompts. It utilizes a natural language understanding model to interpret user prompts and automatically generates a graph configuration for scraping web content.

prompt¶

The user’s natural language prompt for the scraping task.

Type:: str

llm¶

An instance of the ChatOpenAI class configured

Type:: ChatOpenAI

with the specified llm_config.

nodes_description¶

A string description of all available nodes and their arguments.

Type:: str

chain¶

The extraction chain responsible for

Type:: LLMChain

processing the prompt and creating the graph.

build_graph()¶: Executes the graph creation process based on the user prompt

and returns the graph configuration.

convert_json_to_graphviz(json_data)¶: Converts a JSON graph configuration

to a Graphviz object for visualization.

Parameters:

prompt (str) – The user’s natural language prompt describing the desired scraping operation.
url (str) – The target URL from which data is to be scraped.
llm_config (dict) – Configuration parameters for the language model, where ‘api_key’ is mandatory, and ‘model_name’, ‘temperature’, and ‘streaming’ can be optionally included.

Raises:

ValueError – If ‘api_key’ is not included in llm_config.

build_graph()¶

Executes the graph creation process based on the user prompt and: returns the graph configuration.

Returns:: A JSON representation of the graph configuration.
Return type:: dict

static convert_json_to_graphviz(json_data, format: str = 'pdf')¶

Converts a JSON graph configuration to a Graphviz object for visualization.

Parameters:: json_data (dict) – A JSON representation of the graph configuration.
Returns:: A Graphviz object representing the graph configuration.
Return type:: graphviz.Digraph

Module contents¶

This module contains the builders for constructing various components in the ScrapeGraphAI application.