inflow-logo

Inflow: eCommerce Marketing Agency

  • Services
    • SEO
    • PPC
    • CRO
    • Paid Social
  • Clients
    • Case Studies
  • About Us
    • Contact Us
  • Insights Blog
  • Request Proposal
  • Services
    • SEO
    • PPC
    • CRO
    • Paid Social
  • Clients
    • Case Studies
  • About Us
    • Contact Us
  • Insights Blog
  • Request Proposal

Home > eCommerce Digital Marketing Blog > SEO > Technical SEO > How to Scrape eCommerce Sites for SEO Content Audits with Screaming Frog

How to Scrape eCommerce Sites for SEO Content Audits with Screaming Frog

Posted By Mike Belasco on November 24, 2015

  • 15shares

When performing an eCommerce Content Audit, many times our Analysts will require the specific category page content and product page excerpts, so they can analyze for length, originality and more. Our current “go to” tool for crawling websites is Screaming Frog. Their somewhat new (summer 2015) “extraction” feature has made grabbing specific snippets of content for later analysis much easier than before.

Before extraction was available, we’d have to either build our crawler with a tool like import.io or Kimono. Or we’d have to request an export of the content from our client (who sometimes was unable to easily provide this), then use VLOOKUP in a spreadsheet to “link” the content data to the data from Screaming Frog, Google Analytics and URL Profiler, as well. Now, with the new extraction feature, we simply have to set it up before we crawl in Screaming Frog — and we have our content extracted all in one step.

Using Screaming Frog and XPath to Extract Product and Category Content

Screaming Frog provides three different ways to extract content from a Web page: CSSPath, XPath and RegEx. Don’t worry if you are not familiar with these technologies just yet. For our purpose, CSSPath and Xpath are usually the simplest to work with, and by using Chrome Developer tools you won’t have to learn any code.

Step 1: Inspect the Element Where Your Content Resides

Go to a category page like https://adcohearing.com/categories/tv-amplifiers. Right click in the area of the category content and select “Inspect Element” from the menu. Chrome Developer Tools will open up in your browser. This works the same for a product page as well.

TV Amplifiers - Inspect Element

Step 2: Copy the XPath

Highlight the content in the code and right click again. This time highlight “Copy XPath.”

Copy XPath

Step 3: Set Up Your Extraction in Screaming Frog

Go to the “Extractions” setup under the “Custom” menu option. Paste in your Xpath copied from Chrome Developer tools. Choose “XPath” as the type and choose “Extract Text.”

Screaming Frog extract text

Step 4: Run Screaming Frog

Then check out the “Custom Tab in Screaming Frog. Make sure you have “Extraction” selected.

Screaming Frog Custom Extraction

Step 5: Export the Screaming Frog data

You can now export the extracted content along with the rest of your crawl data, including Google Analytics (if you set the tool up to pull that data with your crawl).

Now that you understand the basics of how to find the XPath to an “element” on a Web page and how to use that Xpath to extract the content of that element via Screaming Frog, the possibilities of how to apply this are endless.

What other creative ways have you found to use the extraction tool in Screaming Frog? Let us know in the comments below!

Want more great tips and tools to take control of your eCommerce site? Get access to our eCommerce Content Audit Toolkit. 

eCommerce Content Audits Toolkit
  • 15shares

0 Comments on "How to Scrape eCommerce Sites for SEO Content Audits with Screaming Frog"

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Posts

  • Our reviews of the best SEO Crawl Tools for Large eCommerce Sites Crawlers are essential tools in the SEO and Inbound Marketing world; they’re used for a variety of important projects, from technical audits to comprehensive reviews of the content on a […]
  • How to Find, Assess & Fix Internal Crawl Errors and Broken Redirects Internal crawl errors and unnecessary redirects affect user experience and crawlability - thus your rankings. Checking for these issues is an important part of reviewing past SEO efforts […]
  • The Stalker (2 of 9 Winning Plays) – Screaming Frog Reports for Googlebot Mobile Did you know Google will soon be giving more weight to what their mobile bot finds on your website than what’s found by the desktop-oriented Googlebot they’ve used to crawl the web for […]
Mike Belasco on hiking trail.

Mike Belasco

Mike Belasco has been an entrepreneur and digital marketer since 2003. He’s been the CEO of Inflow (founded as seOverflow) since 2007 and has led Inflow to five Denver’s Fastest-Growing Private Company awards and three Inc. 5000 awards. In 2009, he also founded ConversionIQ, which was acquired by Inflow in 2014.

View Author’s Profile

Related Categories

  • Content Marketing (13)
  • Link Building (16)
  • On-Page SEO (14)
  • SEO Strategy (10)
  • Technical SEO (33)
  • Most Popular Posts:

    eCommerce Marketing Automations Systems Compared
    Technical Mobile Best Practices for SEO and Usability
    Expanding the Horizons of eCommerce Content Strategy
    Thin & Duplicate Content: eCommerce SEO
    5 Ways eCommerce Content Audits Can Increase Revenue
    Want to get content like this straight to your inbox? Subscribe to our weekly content alerts and monthly Inflow Insights newsletter now.

    Categories

    • SEO
      • Content Marketing
      • Link Building
      • On-Page SEO
      • SEO Strategy
      • Technical SEO
    • Paid Advertising
      • Goal Metrics and Analytics
      • Paid Search
      • Paid Search Shopping
      • Paid Social
    • Conversion Rate Optimization
      • A/B Testing
      • eCommerce Page CRO
      • Mobile Conversion Optimization
      • Tools and Plugins
      • Usability
    • Case Studies
    • eCommerce Strategy
      • KPIs and Reporting
    • Digital Marketing Trends in eCommerce
    • Inflow News

    Request a Proposal

    We'll build a custom proposal to meet your goals. Get the process started now.

    Google Premier PartnerInflow is a facebook-certified-creative-strategy-professional Moz Recommended Company Inc 5000 Inflow Clutch Profile
    • Services
      • SEO
      • PPC
      • Conversion
    • Case Studies
      • SEO
      • PPC
      • Conversion
    • Insights Blog
    • Resources
    • More
      • Contact
      • Careers
      • Press Info
      • Privacy Policy
    REQUEST A PROPOSAL
     
    CALL US AT 303-905-1504
    Monday - Friday, 8 a.m. - 6 p.m. (MST)
     
    facebook twitter linked-in linked-in rss

    Send this to a friend