This website uses cookies to ensure you get the best possible experience. See our Cookies Policy.

PMG Digital Made for Humans

Tutorial: How to Crawl a YouTube Channel

3 MINUTE READ | February 3, 2015

Tutorial: How to Crawl a YouTube Channel

What you will need for this tutorial:

  • Scream Frog SEO Spider (free version is available)

  • Text Editor such as Notepad

  • Firefox, Chrome, or Opera Browser

  • Excel


If you’re ever doing an audit on your client’s YouTube channel, crawling it can really come in handy to check extract titles, title length, and description, especially if the channel consists of hundreds of videos. However, crawling a YouTube channel isn’t as easy as just entering the channel URL into your favorite web crawler since most of the videos are inserted into the page using JavaScript, so only a portion of the videos will be successfully crawled.

Crawling a YouTube channel makes it easier to review the video titles–after all, they are the number one driver of click through rate. You can also sort the titles based on character length to see which titles are too long and which titles could use some beefing up. Screaming frog will also pull in the video descriptions BUT only the first 160 characters. But that’s okay, since the most important information about your video should be within in the first few sentences any way. Lastly, the video tags are inserted into the meta keyword tag which can be picked up Screaming Frog. Crawling a competitors YouTube channel can be a great way to see which tags they are using for their videos that your videos may be missing.

Open preferred browser (Firefox, Chrome, or Opera) and download the Web Developer extension. After that is done, restart your browser and go to YouTube and pick a channel. On the channels main page, click videos.

click video link

Depending on whether or not the Channel has it’s videos divided into sections, you will either need select ‘upload’ or ‘all videos’ in the top left drop down menu. If the videos are being displayed in grid-view, you will need to change the display to list view.

click list view

Once in list view, you will need to scroll to the bottom of the page and click “load more” continuously until all of the videos have loaded on the page.

Now that all of your links are displayed on the page, in the web developer extension toolbar in your browser, click information > view link information.

click view link information

Copy and paste the links into a text editor, such as Notepad. Save the list as a text document.

copy URLs

paste urls
  1. Open Screaming Frog. Go to Mode > List. Upload your list of URLs.

  2. Go to Configuration > Spider > Limits and make sure ‘0’ is selected.

  3. Click Start to begin the crawl.

Go forth and conquer. Export into excel and have fun sorting and filtering this information however you would like.

Insights meet inbox

Sign up for weekly articles & resources.

NOTE: You will end up with a few links that are not videos. If you export your crawl into Excel, you can easily get rid of these extraneous links by applying a text filter to the Address column of “does not contain:watch?v=“.


Posted by Kara Eccleston

Related Content

thumbnail image

Get Informed

Video and Streaming Take Center Stage at 2021 IAB NewFronts

6 MINUTES READ | May 14, 2021

Get Informed

The Year of Streaming, and the Decline of Communal Experiences

7 MINUTES READ | March 17, 2021

Get Informed

Oprah Interview Turns Cultural Moment

3 MINUTES READ | March 9, 2021

thumbnail image

Get Informed

Hindsight 2020 & Looking Ahead to 2021

8 MINUTES READ | December 16, 2020

Get Informed

Preparing for Streaming’s Growth & The Future of TV Buying

7 MINUTES READ | December 11, 2020

Get Informed

Athleta Honors Women Everywhere in New Post-Election Ad Campaign

2 MINUTES READ | November 10, 2020

All POST