1,200-Page Content Audit Automated with AI

2024-10-01

1,200+ pages crawled, classified, and prioritized. No spreadsheet.

Built an automated content audit pipeline for a large ecommerce store. Pages crawled, content quality scored, SEO issues classified, and a prioritized action plan generated. No manual review.

The challenge

A large ecommerce store had over 1,200 indexed pages with no clear picture of which were performing, which were thin, which were duplicating content, and which needed to be removed or consolidated. Auditing this manually would take weeks. The goal was to build an automated pipeline that could crawl, classify, and prioritize the full site in a single run.

Approach

• Built a crawl pipeline using Python to systematically pull all indexed pages from the store
• Each page classified by type: product, collection, blog, landing page, thin content, near-duplicate
• Content quality scored per page: word count, heading structure, meta completeness, internal link count
• SEO issues flagged automatically: missing meta descriptions, duplicate title tags, orphaned pages, slow load signals
• Pages ranked by priority: high-impact fixes identified by combining traffic potential with issue severity
• Full output structured as a prioritized action plan, ready to act on

Results

• 1,200+ — pages crawled and classified in one run
• Automated — classification by page type and content quality
• Prioritized — action plan output, ready to act on
• 0 — manual page-by-page review needed

Tools used

Python, Claude AI, Web Crawling, SEO Analysis, Content Classification