Change crawl depth on intranet

Good morning, I use Yacy as a pure intranet search for me. I have on a virtualized Ubuntu server 20.04 (which runs under Proxmox 7.*) a USB hard drive (NTFS). This is mounted there. On the mount directory I run the webserver Lighttp (, which shows me the file structure as http page (similar to FTP), i.e. under the root directory of the webserver is accessible. With Yacy I then crawl, which also works very well.

But I notice that it does not go deeper than 3 directories, i.e. often I see only the 3 directories, although there is another directory with e.g. 2 PDFs below.

What do I have to set where so that EVERYTHING on the USB hard drive is crawled here?

Kind regards - H-BLOGX

How do you start the crawl? If you actively increase the depth of the crawl, you should get also more than 3 directories deep.

Hi Orbiter,

Sorry for the late feedback. Yacy is just an addon that I’m trying to get running cleanly. I deleted everything again, went to “Production” => “Advanced Crawler” and entered as “Start Point”. At “Crawler Filter” I have entered 0 at “Crawling Depth”. At “Unlimited crawl depth for URLs matching with” I also entered Then pressed “Start New Crawl Job” at the bottom.

Under “Administration” => “Process Scheduler” I set the “Event Trigger” to “run regular at 07:00 h” (every morning at 7 o’clock).

If I interpret it correctly, the “Crawling Depth” means the number of continuing domains, right?

I would like to have ONLY my files indexed in my intranet. If there is e.g. in a Word or PDF document somewhere in it, Yacy should NOT switch to Amazon and continue crawling here!

But I want ALL my intranet directories and ALL files under them to be indexed.

The whole thing should be updated once a day. Access should be from a cache because of speed, but it should be ensured during daily crawling that e.g. deleted or moved files are then no longer in the cache, but are re-indexed accordingly.

What do I still have to set everything. Sorry am here still really the layer, but with the old wiki of Yacy I do not get along at all.

Kind regards - H-BLOGX