Rate Limiting a Crawler Across Node Cluster Workers

Code: Megapixel99/webCrawler This project is a small search engine I wrote to understand how one works: a crawler, an inverted index in MongoDB, and a BM25 ranker, all written by hand instead of reach...