Webboard: Fastest Start

[b***@mnogosearch.org]

Author: Alexander Barkov
Email: ***@mnogosearch.org
Message:
Hi,

Hi,
which is the fastest way to import 250.000 URLs and only index the
first page of each URL?

Add this into indexer.conf:

Realm page *

Then import the URLs:

indexer -i -f list.txt

(where list.txt is a file with one URL per line)

Then run indexer in the usual way to crawl and index.

Thanks

Reply: <http://www.mnogosearch.org/board/message.php?id=21636>