aary@xn--e1aghfa.xn--e1aqbccjfc.xn--p1acf to Technology@lemmy.ml · 1 month agoI want to find users of Heritrix3 enginemessage-squaremessage-square1fedilinkarrow-up14arrow-down10file-text
arrow-up14arrow-down1message-squareI want to find users of Heritrix3 engineaary@xn--e1aghfa.xn--e1aqbccjfc.xn--p1acf to Technology@lemmy.ml · 1 month agomessage-square1fedilinkfile-text
I have such setup, but with some problems: it always does full crawl (doesn’t do deduplication) I am unable to control crawl order (don’t have enough knowledge)
minus-squareВежливый_Лис@xn--e1aghfa.xn--e1aqbccjfc.xn--p1acflinkfedilinkarrow-up2·1 month agokreuzcrawl — a high-performance Rust web crawling engine.
kreuzcrawl — a high-performance Rust web crawling engine.