This is done by setting up a loop of text list. Search a website with different keywords and capture the search results (using a Text List)įor anyone that wants to search and extract, you will need to provide Octoparse with a list of keywords to search for. Set up a crawler to first capture URL’s of all the webpages sharing similar web structure, then build a second crawler to visit and extract from each individual URL on the list following the same set of configuration ( learn how).Change a variable list into a fixed list ( learn how).If you need to speed up an extraction, you can: On the contrary, fixed list and URL list are both split-able hence, consider manually changing a variable list into a fixed list or URL list if you need to split a task for faster extraction. Speeding up an extraction with a split-able list (using a Fixed List/URL List)Ī variable list follows a single XPath and matches any elements (as many as there are) that meets the criteria defined by the XPath on a webpage.
Extract from multiple URL’s with similar page layout.Search with multiple keywords on any websites then extract the search results.Speed up an extract task by splitting it.In this article, however, I will like to point out a few scraping scenarios in which you may want to manually switch from one mode to another.Ĭonsider manually selecting or switching a loop mode if you want to: Have you ever built a list in Octoparse? Have you noticed a loop mode gets automatically selected as a loop is created? This may have become so sneaky that you won’t even notice it after a long time of using it.
0 kommentar(er)
