What Is Data Scraping?

Have you ever wondered how websites collect massive amounts of information? Or how businesses extract valuable insights from the vast expanse of the internet? Look no further – we've got you covered in this comprehensive 5-minute guide to data scraping.

By the end of this video, you'll have a solid understanding of what data scraping is, the best tools and techniques and anything you should be careful of.

If you have any questions you would like answered as part of our 'So' series, please get in touch and let us know.

Our podcast, 'The Digital Marketing Podcast' is now in video format on our YouTube channel. Make sure to subscribe for a brand new video podcast every week!

Transcription:

 So, what is data scraping?

Data scraping is the process of taking information from existing websites and places online and then pulling it together. It’s a way of bringing information together that you can't access necessarily very easily or putting it into a way that you can then manipulate it and that's the important bit going forward.

So, how can you then use this data?

Say I want a list of the top 100 grossing movies of all time, and I want to know who the directors were, who the lead actor was and how much money it made. Okay, so I would go and look and if that's on a web page, I can't do anything with it.

Whereas if I can scrape that data off, I can then put it into Excel and I can manipulate it. I could put it into Power BI or a tool like Looker Studio, which is what Google Data Studio is now called. I can manipulate it and visualize it and things like that. What's interesting is that suddenly with things like ChatGPT, I can go through and say, analyze this data for me and manipulate this data and so on as well.

So suddenly I can start mixing these things up, but I need to get the data in the first place. So that's, that's what we can start to do with it.

So, what about tools for helping you do this?

So there's a great Chrome extension called Data Miner. And with Data Miner, you give it a recipe. You would go through and say, okay, here's the webpage. This bit's the title, this bit's the value etc. and you go through and it'll understand the page, click go, and it'll go straight through, thousands and thousands of lines if you want to do. It can go to the next page, and you could go through a whole series of pages, and it'll put that data into a spreadsheet for you. So that's one way of doing it.

The other way is now ChatGPT, because of having browsing built into it, and the plugins built in, you can say, go to this website and get this data and you can get it to pull it for you, and then you can use another plugin to visualize it. So you can start to do some really exciting things between something like DataMiner and ChatGPT with plugins and so on, to kind of get insights into data that you might not have had otherwise.

There's some quite interesting examples of tools within SEO that'll go out and they'll scrape your competitors websites and tell you what's changed. You know, things like that are quite useful to see, like, keep an eye on the competition and see, well, what, what are they updating? What new articles that they published, what's gone out and through social media.

So I think definitely check out some of those, tools like competitor.app. Very, very good for that.

So, is there anything to be careful of?

Yeah, a lot of this stuff will be copyright. You can't just take other people's stuff and use it elsewhere.

You can take it and analyse it and read it and all those kind of things, so you might find it's good for getting insights, but be really careful if you're scraping other people's stuff and then reusing it, there's all sorts of copyright issues you could potentially get into as well, so you've got to be really careful of that.

There are data scraping tools that will go out there and find content that's been plagiarized, and they'll check across like thousands of like different references. One of the common forms of this is when people have scraped photos.

If you scrape an article and publish it on your website and it's using a photo that you don't have permission for, you know, any one of the big photo libraries can slap you with an eight or nine hundred pound fine. And you'll find it difficult not to pay it.  That’s per infringement by the way. So if you've got a data scraping tool that's pulled a whole bunch of images, you're in a whole world of pain. And with AI generated images now, there's no need to be doing this kind of stuff anymore anyway.

So, are there any other good resources?

Yeah, if you just Google “what is data scraping” you will find our article on targetinternet.com at the top there, and you've got a whole step by step guide, you've got some example videos, you've got a load of resources that will help you out. Specifically written for marketers and how they might want to use it.