PMG Digital Made for Humans

Tutorial: How to Crawl a YouTube Channel

3 MINUTE READ | February 3, 2015

Tutorial: How to Crawl a YouTube Channel

Author's headshot

Kara Eccleston

Kara Eccleston has written this article. More details coming soon.

What you will need for this tutorial:

  • Scream Frog SEO Spider (free version is available)

  • Text Editor such as Notepad

  • Firefox, Chrome, or Opera Browser

  • Excel

If you’re ever doing an audit on your client’s YouTube channel, crawling it can really come in handy to check extract titles, title length, and description, especially if the channel consists of hundreds of videos. However, crawling a YouTube channel isn’t as easy as just entering the channel URL into your favorite web crawler since most of the videos are inserted into the page using JavaScript, so only a portion of the videos will be successfully crawled.

Crawling a YouTube channel makes it easier to review the video titles–after all, they are the number one driver of click through rate. You can also sort the titles based on character length to see which titles are too long and which titles could use some beefing up. Screaming frog will also pull in the video descriptions BUT only the first 160 characters. But that’s okay, since the most important information about your video should be within in the first few sentences any way. Lastly, the video tags are inserted into the meta keyword tag which can be picked up Screaming Frog. Crawling a competitors YouTube channel can be a great way to see which tags they are using for their videos that your videos may be missing.

Open preferred browser (Firefox, Chrome, or Opera) and download the Web Developer extension. After that is done, restart your browser and go to YouTube and pick a channel. On the channels main page, click videos.

click video link

Depending on whether or not the Channel has it’s videos divided into sections, you will either need select ‘upload’ or ‘all videos’ in the top left drop down menu. If the videos are being displayed in grid-view, you will need to change the display to list view.

click list view

Once in list view, you will need to scroll to the bottom of the page and click “load more” continuously until all of the videos have loaded on the page.

Now that all of your links are displayed on the page, in the web developer extension toolbar in your browser, click information > view link information.

click view link information

Copy and paste the links into a text editor, such as Notepad. Save the list as a text document.

copy URLs

paste urls
  1. Open Screaming Frog. Go to Mode > List. Upload your list of URLs.

  2. Go to Configuration > Spider > Limits and make sure ‘0’ is selected.

  3. Click Start to begin the crawl.

Go forth and conquer. Export into excel and have fun sorting and filtering this information however you would like.

Stay in touch

Bringing news to you

Subscribe to our newsletter

By clicking and subscribing, you agree to our Terms of Service and Privacy Policy

NOTE: You will end up with a few links that are not videos. If you export your crawl into Excel, you can easily get rid of these extraneous links by applying a text filter to the Address column of “does not contain:watch?v=“.