Subscribe to DSC Newsletter

I'm thinking of developing an application to be run on thousands of individual computers, in the background, a bit like a botnet, but a friendly version of a botnet. Each participant would accept to install a script on their machine, and they would know exactly what the script is doing - essentially information retrieval with a (distributed) web crawler to identify online fraud, browsing web sites and using text mining techniques to identify associations.

For instance, a user would have his/her machine check 200 web pages every day (in the background), and report statistics to us about the domains being visited. This information would be used to score domains, and eliminate fraud.

Are there any products currently offering this functionality?

Views: 122

Reply to This

Replies to This Discussion

I don't know of any for web crawling specifically, but it sounds a lot like SETI -- getting lots of small computers to do data analysis.


On Data Science Central

© 2020 is a subsidiary and dedicated channel of Data Science Central LLC   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service