Search results for "crawlers"

Hits ?▲	Authors	Title	Venue	Year	Link	Author keywords
195	Hai Dong, Farookh Khadeer Hussain, Elizabeth Chang 0001	State of the Art in Semantic Focused Crawlers.	ICCSA (2)	2009	DBLP DOI BibTeX RDF	semantic focused crawlers, ontology-based focused crawlers, metadata abstraction focused crawlers, semantic web, focused crawlers
109	Hai Dong, Farookh Khadeer Hussain, Elizabeth Chang 0001	A survey in semantic web technologies-inspired focused crawlers.	ICDIM	2008	DBLP DOI BibTeX RDF
106	Padmini Srinivasan, Filippo Menczer, Gautam Pant	A General Evaluation Framework for Topical Crawlers.	Inf. Retr.	2005	DBLP DOI BibTeX RDF	evaluation, efficiency, tasks, precision, recall, Web crawlers, topics
106	Filippo Menczer, Gautam Pant, Padmini Srinivasan	Topical web crawlers: Evaluating adaptive algorithms.	ACM Trans. Internet Techn.	2004	DBLP DOI BibTeX RDF	evaluation, reinforcement learning, Efficiency, evolution, exploration, exploitation, topical crawlers
87	C. Lee Giles, Yang Sun 0002, Isaac G. Councill	Measuring the web crawler ethics.	WWW	2010	DBLP DOI BibTeX RDF	robots.txt, web crawler ethics, privacy, ethicality
69	Márcio L. A. Vidal, Altigran Soares da Silva, Edleno Silva de Moura, João M. B. Cavalcanti	GoGetIt!: a tool for generating structure-driven web crawlers.	WWW	2006	DBLP DOI BibTeX RDF	web crawlers, web data extraction, tree edit distance
69	Junghoo Cho, Hector Garcia-Molina	Effective page refresh policies for Web crawlers.	ACM Trans. Database Syst.	2003	DBLP DOI BibTeX RDF	page refresh, world-wide web, web search engines, Web crawlers
68	Gautam Pant, Padmini Srinivasan	Learning to crawl: Comparing classification schemes.	ACM Trans. Inf. Syst.	2005	DBLP DOI BibTeX RDF	machine learning, classifiers, focused crawlers, Topical crawlers
62	Duen Horng Chau, Shashank Pandit, Samuel Wang, Christos Faloutsos	Parallel crawling for online social networks.	WWW	2007	DBLP DOI BibTeX RDF	parallelization, online social networks, web crawler, web spider
62	Andres E. Mora Vargas, Kenzuke Mizuuchi, Daisuke Endo, Eric Rohmer, Keiji Nagatani, Kazuya Yoshida	Development of a Networked Robotic System for Disaster Mitigation, -Navigation System based on 3D Geometry Acquisition.	IROS	2006	DBLP DOI BibTeX RDF
59	Gautam Pant, Padmini Srinivasan	Link Contexts in Classifier-Guided Topical Crawlers.	IEEE Trans. Knowl. Data Eng.	2006	DBLP DOI BibTeX RDF	performance evaluation, Web Search, Web mining
50	Franziska von dem Bussche, Klara A. Weiand, Benedikt Linse, Tim Furche, François Bry	Not so creepy crawler: easy crawler generation with standard xml queries.	WWW	2010	DBLP DOI BibTeX RDF	xml, data extraction, web crawler, web query
50	Akiya Kamimura	A Novel Mobile Robot Design for High Maneuverability.	IROS	2006	DBLP DOI BibTeX RDF
50	Demetrios Zeinalipour-Yazti, Marios D. Dikaiakos	Design and Implementation of a Distributed Crawler and Filtering Processor.	NGITS	2002	DBLP DOI BibTeX RDF
47	Yong Wang, Yiqun Liu 0001, Min Zhang 0006, Shaoping Ma	News Page Discovery Policy for Instant Crawlers.	AIRS	2008	DBLP DOI BibTeX RDF	news page discovery, web log, user behavior analysis
46	Márcio L. A. Vidal, Altigran Soares da Silva, Edleno Silva de Moura, João M. B. Cavalcanti	Structure-driven crawler generation by example.	SIGIR	2006	DBLP DOI BibTeX RDF	digital libraries, web crawlers, tree edit distance
44	Jesper Bergman, Oliver Popov	Exploring Dark Web Crawlers: A Systematic Literature Review of Dark Web Crawlers and Their Implementation.	IEEE Access	2023	DBLP DOI BibTeX RDF
43	Ziyu Guan, Can Wang 0001, Chun Chen 0001, Jiajun Bu, Junfeng Wang	Guide focused crawler efficiently and effectively using on-line topical importance estimation.	SIGIR	2008	DBLP DOI BibTeX RDF	focused crawlers, topical crawlers
43	Andrei Z. Broder, Marc Najork, Janet L. Wiener	Efficient URL caching for world wide web crawling.	WWW	2003	DBLP DOI BibTeX RDF	URL caching, distributed crawlers, web graph models, caching, crawling, web crawlers
37	Jason J. Jung	Using evolution strategy for cooperative focused crawling on semantic web.	Neural Comput. Appl.	2009	DBLP DOI BibTeX RDF	Cooperative focused crawler, Semantic web, Evolution strategy
37	Weifeng Zhang 0001, Baowen Xu, Hong Lu	Web Page's Blocks Based Topical Crawler.	SOSE	2008	DBLP DOI BibTeX RDF
37	Milly Kc, Markus Hagenbuchner, Ah Chung Tsoi	A Scalable Lightweight Distributed Crawler for Crawling with Limited Resources.	Web Intelligence/IAT Workshops	2008	DBLP DOI BibTeX RDF
37	Yuan Wan, Hengqing Tong	URL Assignment Algorithm of Crawler in Distributed System Based on Hash.	ICNSC	2008	DBLP DOI BibTeX RDF
37	Luciano Barbosa, Juliana Freire	An adaptive crawler for locating hiddenwebentry points.	WWW	2007	DBLP DOI BibTeX RDF	HiddenWeb, web crawling strategies, online learning, learning classifiers
37	Guilherme T. de Assis, Alberto H. F. Laender, Marcos André Gonçalves, Altigran Soares da Silva	Exploiting Genre in Focused Crawling.	SPIRE	2007	DBLP DOI BibTeX RDF	Focused crawling, Web crawling, SVM classifiers
37	Carlos Castillo 0001, Alberto Nelli, Alessandro Panconesi	A Memory-Efficient Strategy for Exploring the Web.	Web Intelligence	2006	DBLP DOI BibTeX RDF
37	Jialun Qin, Yilu Zhou, Michael Chau	Building domain-specific web collections for scientific digital libraries: a meta-search enhanced focused crawling method.	JCDL	2004	DBLP DOI BibTeX RDF	domain-specific collection building, web search algorithm, digital libraries, focused crawling, meta-search
37	Wang Lam, Hector Garcia-Molina	Multicasting a Changing Repository.	ICDE	2003	DBLP DOI BibTeX RDF
37	Joel L. Wolf, Mark S. Squillante, Philip S. Yu, Jay Sethuraman, L. Ozsen	Optimal crawling strategies for web search engines.	WWW	2002	DBLP DOI BibTeX RDF
35	Shuyi Zheng, Pavel A. Dmitriev, C. Lee Giles	Graph-based seed selection for web-scale crawlers.	CIKM	2009	DBLP DOI BibTeX RDF	pagerank, crawler, graph analysis, seed selection
35	Mauricio Marín, Rodrigo Paredes, Carolina Bonacic	High-performance priority queues for parallel crawlers.	WIDM	2008	DBLP DOI BibTeX RDF	priority queues, parallel and distributed computing
35	Frank McCown, Joan A. Smith, Michael L. Nelson	Lazy preservation: reconstructing websites by crawling the crawlers.	WIDM	2006	DBLP DOI BibTeX RDF	cached resources, search engine, recovery, digital preservation
35	Chris McIntosh, Ghassan Hamarneh	Vessel Crawlers: 3D Physically-based Deformable Organisms for Vasculature Segmentation and Analysis.	CVPR (1)	2006	DBLP DOI BibTeX RDF
35	Anália Lourenço, Orlando Belo	Catching web crawlers in the act.	ICWE	2006	DBLP DOI BibTeX RDF	data webhousing, web usage mining, web crawler, traverse patterns
35	Chris McIntosh, Ghassan Hamarneh	Spinal Crawlers: Deformable Organisms for Spinal Cord Segmentation and Analysis.	MICCAI (1)	2006	DBLP DOI BibTeX RDF
35	Duo Zhang, Yan Qiu Chen	Classifying Image Texture with Artificial Crawlers.	IAT	2004	DBLP DOI BibTeX RDF
35	Xiaoming Liu 0005, Kurt Maly, Mohammad Zubair, Michael L. Nelson	DP9: an OAI gateway service for web crawlers.	JCDL	2002	DBLP DOI BibTeX RDF	gateway service, deep web, open archives initiative
35	Ricardo A. Baeza-Yates, José M. Piquer	Agents, Crawlers, and Web Retrieval.	CIA	2002	DBLP DOI BibTeX RDF
34	Jun Lai, Ben Soh, Chai Fei	A Web Page Ranking Method by Analyzing Hyperlink Structure and K-Elements.	ICCSA (4)	2006	DBLP DOI BibTeX RDF	Information retrieval, World Wide Web, search engine, hyperlinks, keywords, web crawlers, elements
25	Myriam Ben Saad, Stéphane Gançarski	Using visual pages analysis for optimizing web archiving.	EDBT/ICDT Workshops	2010	DBLP DOI BibTeX RDF	visual page analysis, change detection, web archiving, web crawling
25	Guilherme T. de Assis, Alberto H. F. Laender, Marcos André Gonçalves, Altigran Soares da Silva	A Genre-Aware Approach to Focused Crawling.	World Wide Web	2009	DBLP DOI BibTeX RDF	document genre exploitation, focused crawling, web crawling
25	Andrzej Sieminski	Using WordNet to Measure the Similarity of Link Texts.	ICCCI	2009	DBLP DOI BibTeX RDF	focused browsers, word semantic similarity, link text selection, WordNet
25	Steve Webb, James Caverlee, Calton Pu	Predicting web spam with HTTP session information.	CIKM	2008	DBLP DOI BibTeX RDF	HTTP session information, classification, web spam
25	Zhumin Chen, Jun Ma 0001, Xiaohui Han, Dongmei Zhang 0002	An Effective Relevance Prediction Algorithm Based on Hierarchical Taxonomy for Focused Crawling.	AIRS	2008	DBLP DOI BibTeX RDF	Hierarchical Topic Taxonomy, Topic Description, Focused Crawling, Relevance Prediction
25	Dheerendranath Mundluru, Xiongwu Xia	Experiences in crawling deep web in the context of local search.	GIR	2008	DBLP DOI BibTeX RDF	deep web crawling, local search, wrappers, structured data
25	Rui Chen 0012, Bipin C. Desai, Cong Zhou	CINDI Robot: an Intelligent Web Crawler Based on Multi-level Inspection.	IDEAS	2007	DBLP DOI BibTeX RDF	focused web crawler, Naïve Bayes classifier, multi-level inspection, revised context graph, tunneling, SVM classifier
25	José Exposto, Joaquim Macedo 0001, António Pina, Albano Alves, José Rufino	Efficient Partitioning Strategies for Distributed Web Crawling.	ICOIN	2007	DBLP DOI BibTeX RDF
25	Jason J. Jung, GeunSik Jo, Seong Won Yeo	Meta-evolution Strategy to Focused Crawling on Semantic Web.	ICANN (2)	2007	DBLP DOI BibTeX RDF
25	Hitoshi Kimura, Fumihiro Kajimura, Daisuke Maruyama, Michihiko Koseki, Norio Inou	Flexible Hermetically-Sealed Mobile Robot for Narrow Spaces Using Hydrostatic Skeleton Driving Mechanism.	IROS	2006	DBLP DOI BibTeX RDF
25	Daniel Gomes, André L. Santos 0001, Mário J. Silva	Managing duplicates in a web archive.	SAC	2006	DBLP DOI BibTeX RDF
25	Meiyappan Yuvarani, N. Ch. S. N. Iyengar, Arputharaj Kannan	LSCrawler: A Framework for an Enhanced Focused Web Crawler Based on Link Semantics.	Web Intelligence	2006	DBLP DOI BibTeX RDF
25	Vijayalakshmi Amudhan, Devi Thirupathi	Traffic Adaptive Optimum Updating Scheme for Search Engines.	ICDIM	2006	DBLP DOI BibTeX RDF
25	Zsolt Palotai, Csilla Farkas, András Lörincz	Selection in Scale-Free Small World.	CEEMAS	2005	DBLP DOI BibTeX RDF
25	Jaeyoung Yang, Jinbeom Kang, Joongmin Choi	A Focused Crawler with Document Segmentation.	IDEAL	2005	DBLP DOI BibTeX RDF
25	Fei Liu 0007, Fanyuan Ma, Yunming Ye, Minglu Li 0001, Jiadi Yu	IglooG: A Distributed Web Crawler Based on Grid Service.	APWeb	2005	DBLP DOI BibTeX RDF
25	Charu C. Aggarwal	On Leveraging User Access Patterns for Topic Specific Crawling.	Data Min. Knowl. Discov.	2004	DBLP DOI BibTeX RDF	world wide web, querying, crawling
25	Marios D. Dikaiakos, Athena Stassopoulou, Loizos Papageorgiou	Characterizing Crawler Behavior from Web Server Access Logs.	EC-Web	2003	DBLP DOI BibTeX RDF
25	Shoubin Dong, Xiaofeng Lu, Ling Zhang, Kejing He	An Efficient Parallel Crawler in Grid Environment.	GCC (1)	2003	DBLP DOI BibTeX RDF
25	Onn Brandman, Junghoo Cho, Hector Garcia-Molina, Narayanan Shivakumar	Crawler-Friendly Web Servers.	SIGMETRICS Perform. Evaluation Rev.	2000	DBLP DOI BibTeX RDF
22	Mikhail Drobyshevskiy, Denis Aivazov, Denis Turdakov, Alexander Yatskov, Maksim Varlamov, Danil Shayhelislamov	Collecting Influencers: A Comparative Study of Online Network Crawlers.	CoRR	2024	DBLP DOI BibTeX RDF
22	Charles B. Schaff, Audrey Sedal, Shiyao Ni, Matthew R. Walter	Sim-to-real transfer of co-optimized soft robot crawlers.	Auton. Robots	2023	DBLP DOI BibTeX RDF
22	Ruoyao Wang, Peter A. Jansen	Self-Supervised Behavior Cloned Transformers are Path Crawlers for Text Games.	CoRR	2023	DBLP DOI BibTeX RDF
22	Domenico Benfenati, Marco Montanaro, Antonio M. Rinaldi, Cristiano Russo, Cristian Tommasino	Using Focused Crawlers with Obfuscation Techniques in the Audio Retrieval Domain.	MEDES	2023	DBLP DOI BibTeX RDF
22	Ruoyao Wang, Peter A. Jansen	Self-Supervised Behavior Cloned Transformers are Path Crawlers for Text Games.	EMNLP (Findings)	2023	DBLP DOI BibTeX RDF
22	Pawel Drozda, Bartosz A. Nowak, Arkadiusz Talun, Leszek Bukowski	Evaluating Web Crawlers with Machine Learning Algorithms for Accurate Location Extraction from Job Offers.	ICCCI (CCIS Volume)	2023	DBLP DOI BibTeX RDF
22	Ruslan Gabdrahmanov, Tatyana Tsoy, Yang Bai 0006, Mikhail M. Svinin, Evgeni Magid	Gear Wheels based Simulation of Crawlers for Mobile Robot Servosila Engineer.	ICINCO	2022	DBLP DOI BibTeX RDF
22	Shridevi s, Shashwat Sanket, Jayraj Thakor, Dhivya M	Analysis of Semantic and Non-Semantic crawlers.	ISIC	2021	DBLP BibTeX RDF
22	Bhupendra Acharya, Phani Vadrevu	PhishPrint: Evading Phishing Detection Crawlers by Prior Profiling.	USENIX Security Symposium	2021	DBLP BibTeX RDF
22	Tianyi Yue, Yadong Zhou, Bowen Hu, Zhanbo Xu, Xiaohong Guan, Hao Zhou, Ting Liu 0002	An Unsupervised-Learning Based Method for Detecting Groups of Malicious Web Crawlers in Internet.	CASE	2021	DBLP DOI BibTeX RDF
22	Michael Röder, Geraldo de Souza, Denis Kuchelev, Abdelmoneim Amer Desouki, Axel-Cyrille Ngonga Ngomo	ORCA - a Benchmark for Data Web Crawlers.	ICSC	2021	DBLP DOI BibTeX RDF
22	Edi Stambuk, Stjepan Gros, Marin Vukovic	Analyzing Web Security Features using Crawlers: Study of Croatian Web.	ConTEL	2021	DBLP DOI BibTeX RDF
22	Andriy Sultanov, Maksym Protsyk, Maksym Kuzyshyn, Daria Omelkina, Vyacheslav Shevchuk, Oleg Farenyuk	Comparison of performance of the popular approaches to implementing parallel crawlers.	CSIT (1)	2021	DBLP DOI BibTeX RDF
22	Junji Hirasawa	Improvement of the Mobility on the Step-Field for a Stair Climbable Robot with Passive Crawlers.	J. Robotics Mechatronics	2020	DBLP DOI BibTeX RDF
22	Shiju Li, Chul-Ho Lee, Do Young Eun	Trapping Malicious Crawlers in Social Networks.	CIKM	2020	DBLP DOI BibTeX RDF
22	Syed Suleman Ahmad, Muhammad Daniyal Dar, Muhammad Fareed Zaffar, Narseo Vallina-Rodriguez, Rishab Nithyanand	Apophanies or Epiphanies? How Crawlers Impact Our Understanding of the Web.	WWW	2020	DBLP DOI BibTeX RDF
22	Ayaka Watanabe, Masayuki Okugawa, Katsuji Oogane, Tetsuya Kimura, Tetsuya Kinugasa, Yoshikazu Ohtsubo	Effect of Compliance on Ground Adaptability of Crawler Mobile Robots with Sub-Crawlers.	SII	2020	DBLP DOI BibTeX RDF
22	Nafiseh Hosseini, Fatemeh Fakhar, Behzad Kiani, Saeid Eslami	Enhancing the security of patients' portals and websites by detecting malicious web crawlers using machine learning techniques.	Int. J. Medical Informatics	2019	DBLP DOI BibTeX RDF
22	Michael Röder, Geraldo de Souza, Denis Kuchelev, Abdelmoneim Amer Desouki, Axel-Cyrille Ngonga Ngomo	ORCA: a Benchmark for Data Web Crawlers.	CoRR	2019	DBLP BibTeX RDF
22	Shengye Wan, Yue Li 0002, Kun Sun 0001	PathMarker: protecting web contents against inside crawlers.	Cybersecur.	2019	DBLP DOI BibTeX RDF
22	Sergey Kratov	About Leaks of Confidential Data in the Process of Indexing Sites by Search Crawlers.	Ershov Informatics Conference	2019	DBLP DOI BibTeX RDF
22	Weiping Zhu 0004, Hang Gao, Zongjian He, Jiangbo Qin, Bo Han	A Hybrid Approach for Recognizing Web Crawlers.	WASA	2019	DBLP DOI BibTeX RDF
22	Konstantin Avrachenkov, Vivek S. Borkar	A learning algorithm for the Whittle index policy for scheduling web crawlers.	Allerton	2019	DBLP DOI BibTeX RDF
22	Bo Wen	Application of Distributed Web Crawlers in Information Management System.	Informatica (Slovenia)	2018	DBLP BibTeX RDF
22	Inwoo Ro, Joong Soo Han, Eul Gyu Im	Detection Method for Distributed Web-Crawlers: A Long-Tail Threshold Model.	Secur. Commun. Networks	2018	DBLP DOI BibTeX RDF
22	Lakmal Meegahapola, Vijini Mallawaarachchi, Roshan Alwis, Eranga Nimalarathna, Dulani Meedeniya, Sampath Jayarathna	Random Forest Classifier based Scheduler Optimization for Search Engine Web Crawlers.	ICSCA	2018	DBLP DOI BibTeX RDF
22	Weiping Zhu 0004, Jiangbo Qin, Ruoshan Kong, Hai Lin, Zongjian He	A System Framework for Efficiently Recognizing Web Crawlers.	SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI	2018	DBLP DOI BibTeX RDF
22	Bruno Rodrigues Froz, Antonio Oseas de Carvalho Filho, Aristófanes Corrêa Silva, Anselmo Cardoso de Paiva, Rodolfo Acatauassu Nunes, Marcelo Gattass	Lung nodule classification using artificial crawlers, directional texture and support vector machine.	Expert Syst. Appl.	2017	DBLP DOI BibTeX RDF
22	Manish Kumar, Rajesh Kumar Bhatia, Dhavleesh Rattan	A survey of Web crawlers for information retrieval.	WIREs Data Mining Knowl. Discov.	2017	DBLP DOI BibTeX RDF
22	Emin Islam Tatli, Bedirhan Urgun	WIVET - Benchmarking Coverage Qualities of Web Crawlers.	Comput. J.	2017	DBLP DOI BibTeX RDF
22	Justin F. Brunelle, Michele C. Weigle, Michael L. Nelson	Archival Crawlers and JavaScript: Discover More Stuff but Crawl More Slowly.	JCDL	2017	DBLP DOI BibTeX RDF
22	Ritu Sharma, Manik Lal Das	On the verification of conjunctive keyword search results using authenticated crawlers.	COMSNETS	2017	DBLP DOI BibTeX RDF
22	Shengye Wan, Yue Li 0002, Kun Sun 0001	Protecting web contents against persistent distributed crawlers.	ICC	2017	DBLP DOI BibTeX RDF
22	Oussama Beroual, Francis Guerin, Sylvain Hallé	Searching for Behavioural Bugs with Stateful Test Oracles in Web Crawlers.	SBST@ICSE	2017	DBLP DOI BibTeX RDF
22	Apostolis Zarras, Federico Maggi	Hiding Behind the Shoulders of Giants: Abusing Crawlers for Indirect Web Attacks.	PST	2017	DBLP DOI BibTeX RDF
22	Yajun Du, Yong Xu, Min Wang	A Novel Cooperation and Competition Strategy Among Multi-Agent Crawlers.	Comput. Informatics	2016	DBLP BibTeX RDF
22	Karane Vieira, Luciano Barbosa, Altigran Soares da Silva, Juliana Freire, Edleno Silva de Moura	Finding seeds to bootstrap focused crawlers.	World Wide Web	2016	DBLP DOI BibTeX RDF
22	Deepika 0001, Ashutosh Dixit	URL ordering policies for distributed crawlers: a review.	CoRR	2016	DBLP BibTeX RDF
22	Shankar Karuppayah, Emmanouil Vasilomanolakis, Steffen Haas, Max Mühlhäuser, Mathias Fischer 0001	BoobyTrap: On autonomously detecting and characterizing crawlers in P2P botnets.	ICC	2016	DBLP DOI BibTeX RDF
22	Masatoshi Hatano, Yuki Kitahara	Research on controls of sub crawlers for climbing up stairs with LRF for rescue robots.	ICARCV	2016	DBLP DOI BibTeX RDF
22	Adi Omari, Sharon Shoham, Eran Yahav	Cross-supervised synthesis of web-crawlers.	ICSE	2016	DBLP DOI BibTeX RDF