What you will need for this tutorial:
- Scream Frog SEO Spider (free version is available)
- Text Editor such as Notepad
- Firefox, Chrome, or Opera Browser
- Web Developer Extension
Why would I want to crawl a YouTube channel?
Crawling a YouTube channel makes it easier to review the video titles–after all, they are the number one driver of click through rate. You can also sort the titles based on character length to see which titles are too long and which titles could use some beefing up. Screaming frog will also pull in the video descriptions BUT only the first 160 characters. But that’s okay, since the most important information about your video should be within in the first few sentences any way. Lastly, the video tags are inserted into the meta keyword tag which can be picked up Screaming Frog. Crawling a competitors YouTube channel can be a great way to see which tags they are using for their videos that your videos may be missing.
Open preferred browser (Firefox, Chrome, or Opera) and download the Web Developer extension. After that is done, restart your browser and go to YouTube and pick a channel. On the channels main page, click videos.
Depending on whether or not the Channel has it’s videos divided into sections, you will either need select ‘upload’ or ‘all videos’ in the top left drop down menu. If the videos are being displayed in grid-view, you will need to change the display to list view.
Once in list view, you will need to scroll to the bottom of the page and click “load more” continuously until all of the videos have loaded on the page.
Now that all of your links are displayed on the page, in the web developer extension toolbar in your browser, click information > view link information.
Copy and paste the links into a text editor, such as Notepad. Save the list as a text document.
- Open Screaming Frog. Go to Mode > List. Upload your list of URLs.
- Go to Configuration > Spider > Limits and make sure ‘0’ is selected.
- Click Start to begin the crawl.
Go forth and conquer. Export into excel and have fun sorting and filtering this information however you would like.
NOTE: You will end up with a few links that are not videos. If you export your crawl into Excel, you can easily get rid of these extraneous links by applying a text filter to the Address column of “does not contain:watch?v=“.