Q: Is there any speed limit using Local Extraction?
Solutions are available for a related question here. Go to have a check now!
Description:
It took long to scrape 2000 pages using Local Extraction and a URL list. Why?
Is there any speed limit using Local Extraction?
A:
There is no speed limit when using Local Extraction to scrape lots of web pages.
The extraction speed may depend on the factors below:
1) the complexity of the task you configured (the more complex the task is, the slower the extraction).
2) the load speed of the website you want to crawl.
3) the Internet environment.
4) something else like a memory leak, and etc.
Besides the factor mentioned above, you can choose not to load image to speed up the extraction. For those list/table web pages, you can try to use our Smart Mode to get the results directly.
If you have any questions, we'd happy to help.
