#Octoparse create template install

Getting started with Octoparseįirst of all, you need to install Octoparse. Then we will define a scraping task aimed at extracting data from the main table of that webpage.

#Octoparse create template how to

This is a good example of a webpage whose data is updated frequently over time.įirst, we will see how to install Octoparse. Let’s say we want to scrape data from the List_of_countries_and_dependencies_by_population Wikipedia page. They can be used in case of aggressive websites to hide IP and avoid IP blocking. Then, it comes also with an API program, which I will show you how to use shortly.įurthermore, although the tool reproduces human activity to communicate with web pages and avoid being detected while scraping, it offers IP proxy servers as well. Plus, it provides a scheduled cloud extraction feature to extract dynamic data in real-time. Then, data extracted from multiple websites can be easily saved and structured in many formats. In each case, Octoparse involves a user-friendly point-and-click interface conceived to guide you throughout the data extraction process. While the third one is a flexible and powerful mode designed for those requiring more custom needs. The second one is a simple way to scrape data based on a number of pre-built templates employable from anyone with no effort. The first one is based on an auto-detection algorithm designed to automatically scrape pages containing items nested in a list or a table. It offers a large set of features, including auto-detection, task templates, and an advanced mode. Octoparse is a robust website crawler aimed at extracting every kind of data you need from the web. Now you’ve learned how to build your first task.“Octoparse is an extremely powerful data extraction tool that has optimized and pushed our data scraping efforts to the next level” - Octoparse official website Or choose “Cloud Extraction” to run the task in the cloud platform. Then export the results to Excel files, or other formats and save the file to the computer. The data extracted will be shown in the data extracted pane. But sometimes you may cause problems on certain websites.) Then click “Next”.Ĭhoose “Local Extraction” to run the task on your computer. (Note: You can choose not to load image to speed up the extraction. Now you are done configuring extraction rule. Next, drag the second “Loop item” before the “Click to paginate” action. ➜ Select “Extract text”.Īfter selecting “Extract text”, the data you choose will be shown in these fields.

Start extracting data by clicking on the data you want. Now it will navigate to the detail page every time it does the “Loop Item” action.

➜ Select “Finish creating list.” ➜ Click “Loop” to process the list. ➜ Select “Create a list of items” ➜ Select “ Add current item to the list”. Select the first title that links to the detail page to create a list of items. Next, create a list of items to navigate to the detail page. Now the pagination action has been created. ➜ Select “Loop click the element” under Advanced options. ➜ Click “Go” button to open the web page.

First, open the website you plan to crawl in the built-in browser.