ScrapeForge AI

AI web scraping assistant by Sharad Khare.

ScrapeForge AI extracts structured web data from natural-language prompts using ScrapeGraphAI, with both OpenAI cloud mode and local Ollama mode.

Features

• Natural-language web extraction prompts
• OpenAI model mode (app.py) for cloud scraping
• Local Ollama mode (app_local.py) for privacy-focused scraping
• Structured output for products, leads, content, and competitor research
• Streamlit interface for fast scraping workflows

Quick start

cd scrapeforge-ai
pip install -r requirements.txt
streamlit run app.py

Local mode:

streamlit run app_local.py

Required setup

Cloud mode:

• OpenAI API key

Local mode:

• Ollama running at http://localhost:11434

How it works

1. Enter a website URL and extraction prompt.

2. ScrapeGraphAI builds an extraction graph from your instructions.

3. The selected model parses and extracts structured data.

4. Results are returned directly in the Streamlit app.

Project structure

scrapeforge-ai/
├── app.py
├── app_local.py
├── scrapeforge/
│   ├── config.py
│   └── services.py
├── requirements.txt
├── pyproject.toml
└── README.md

Who this is for

• Growth teams collecting structured web intelligence
• Sales teams building lead extraction workflows
• Developers shipping AI-powered web scraping tools
• Founders automating competitor and content research

Use cases

• Extract product catalogs and pricing data
• Pull lead details from business pages
• Convert article pages into structured JSON-like outputs
• Monitor competitor updates and feature changes

Author

• Website: sharadkhare.in

License

MIT © Sharad Khare