[Psycho_Coder]

Hello,

I want to increase my knowledge more on web content extraction and web crawling. I have run out of ideas about what to crawl and what kinds of bots I can make.

I am not sure if I should used the term bot here.

If you have some idea do throw them here.

Thank you,
Sincerely,
Psycho_Coder.

[Deque]

Have a look at these:
http://www.engpaper.com/web-crawler-2012...papers.htm
http://research.microsoft.com/pubs/10293...ecture.pdf
https://evilzone.org/tutorials/analysis-...-spambots/
http://cis.poly.edu/suel/papers/crawl.pdf
http://www.bagualu.net/linux/crawler.pdf
http://support.sas.com/resources/papers/...3-2010.pdf
http://www.ldodds.com/projects/slug/slug...rawler.pdf
http://www.econstor.eu/bitstream/10419/5...55853X.pdf
http://ijcsi.org/papers/IJCSI-8-6-1-309-313.pdf

[Psycho_Coder]

Have a look at these:
http://www.engpaper.com/web-crawler-2012...papers.htm
http://research.microsoft.com/pubs/10293...ecture.pdf
https://evilzone.org/tutorials/analysis-...-spambots/
http://cis.poly.edu/suel/papers/crawl.pdf
http://www.bagualu.net/linux/crawler.pdf
http://support.sas.com/resources/papers/...3-2010.pdf
http://www.ldodds.com/projects/slug/slug...rawler.pdf
http://www.econstor.eu/bitstream/10419/5...55853X.pdf
http://ijcsi.org/papers/IJCSI-8-6-1-309-313.pdf

Now thats what I call a great feedback

[dropzon3]

You can always move into building game bots for browser based mmorpgs. It combines both the scraping aspect and logic aspects as you try to play like a normal player.

[dropzon3]

You can always move into building game bots for browser based mmorpgs. It combines both the scraping aspect and logic aspects as you try to play like a normal player.

[Singularity_mybb_import13102]

I was bored and I created a script that was scrapping entire PHPBB forums and inserting the content into a local database.
I later realized that something went wrong in it, so I have to fix some thing, but it was sort of fun.

I'm thankful for @Deque's feedback, as I'm also looking for new things to do in this field of programming.

[Singularity_mybb_import13102]

I was bored and I created a script that was scrapping entire PHPBB forums and inserting the content into a local database.
I later realized that something went wrong in it, so I have to fix some thing, but it was sort of fun.

I'm thankful for @Deque's feedback, as I'm also looking for new things to do in this field of programming.