Scribd
Upload
11 views

Class 09

Uploaded by

nibirforwork2007
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
11 views

Class 09

Uploaded by

nibirforwork2007
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 12

Class 09

Web Scraping with Developer Tools


 Create Sitemap
 URL Selection
 Add New Selector
 ID / Name creates
 Type selects
 Selector type select / Multiple Select
 Parent Select
 Data Export
How to Work with Web Scraper?
Web Scraper is a user-friendly Chrome
extension that allows you to extract data
from websites without writing complex
code. Here's a basic guide on how to use it:
1. Install the Extension:
 Go to the Chrome Web Store and search for "Web
Scraper."
 Install the extension.
2. Create a New Site Map:
 Open the Web Scraper icon in your Chrome
toolbar.
 Click "Create New Site Map."
 Give your site map a name.
3. Define the Starting URL:
 In the "Start URL" field, enter the URL of the
webpage you want to scrape.

4. Create Selectors:
 A selector is a rule that defines the specific data
you want to extract.
 Click the "Add Selector" button.
 Choose the type of selector:
o Element: For extracting text content from

HTML elements.
o Link: For extracting links from the page.
o Image: For extracting image URLs.

o Input: For extracting input fields (like search

boxes).
 Use the selector builder to target the specific
elements on the page. You can use CSS selectors
or XPath expressions.
 Give your selector a name and optionally add a
description.
5. Configure Site Map Settings:
 You can configure settings like:
o Pagination: To handle paginated websites.

o Delay: To avoid overloading the server.

o Export Format: To choose the output format

(CSV, JSON, XML).


6. Run the Scraper:
 Click the "Start Scraping" button.
 The scraper will follow the defined rules and
extract the specified data.
7. Export the Data:
 Once the scraping is complete, you can export
the extracted data in your chosen format.
Additional Tips:

 Inspect the Page: Use your browser's developer tools to


inspect the HTML structure and identify the elements you
want to scrape.
 Test Your Selectors: Make sure your selectors are accurate
by testing them on a few pages.
 Handle Dynamic Content: For websites that load content
dynamically, you might need to use more advanced
techniques like JavaScript rendering or API scraping.
 Respect Website Terms of Service: Always respect the
website's terms of service and robots.txt file. Avoid
overloading the server and avoid scraping personal data.

By following these steps and understanding the basics of web


scraping, you can effectively use Web Scraper to extract valuable
data from websites.

Practical
 Create Sitemap & URL Selection
1.Go to where a lot of data exists.
Like Real Yellow Page. Then,
Search a keyword like Doctor/Teacher or anything.

2.Open Web developer tools. Select Web Scraper


3.Click Create new Sitemap then click Create Sitemap. Like
this 

4.Set a name and a Link


Collect Web page URL

Set a name

Paste Page URL

Then click OK.


5. Create your sitemap.
Main Page called _root

 Add New Selector

1.Click Add New Selector. Then it shows like this 


2.When it is Done its name is the same as an ID’s name.
 ID, Type, Selection, Multiple Selection,
Parent Selection.
ID, Type, Multiple, Parent Select like this 
1. First ID select an ID like this .

Doctors
2. Then select what type is it. If it’s hyperlink use Like.
3. Besides using only Text.

4. Selection
Click Select if you collect just one Data that’s ok,
But if you need multiple you first click multiple then click
Select.

Select What Data you need but remember the condition is


all Data are same phase.
Then click Done Selection.

And the at last click Save Selector.

Then you see like this 


So, in this way create a Selector

You also create a Selector Child’s but


conditions:
 The selector must be a select-type
Link.
Sitemap  Add selector > Link 
Add selector  then add
 Name

 Address

 E-mail

 Phone Number

Parent-to-Parent Data Copy


Sitemap  Selector
Selector  Edit 
Then Data copy to _root to your
Selector.

Data Export
Follow Instruction
1. Click Sitemap “your selector name” then

2. click Scrape.
3. Then set the timeline and click Start Scraping

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy