On many occasions I see that YaCy ignores sitemap.xml from a site to be crawled.
When I paste the link to start of the crawler, it does not allow me to click on sitemap url.
I think this happens when robots.txt does not have a sitemap entry though the site has a sitemap_index.xml or sitemap.xml files
Another issue is that there are sites that contain multiple sitemap files, i.e.
Sitemap: https://www.cnn.gr/sitemap-news
Sitemap: https://www.cnn.gr/sitemap/articles
etc.
And yacy loads only the first one in such cases.
Is there a way to manually enforce one or more sitemap urls in these situations?
Thanks
Ian