need a help - Printable Version

need a help - Printable Version

+- Sinisterly (https://sinister.li)
+-- Forum: Coding (https://sinister.li/Forum-Coding)
+--- Forum: Visual Basic & .NET Framework (https://sinister.li/Forum-Visual-Basic-NET-Framework)
+--- Thread: need a help (/Thread-need-a-help)

need a help - polas - 03-09-2012

Hi everyone i have not small and not big problem i do not know if it's happening and for others but
the problem is that that some times i use my google scraper to extract links from google search engine and it works but some times i get windows form in vb.net application is not responding
and not even that.
And this is happing for a long time and i do not know what is wrong with my codes.
I do not get any errors or something and when try to open internet browser it drops me sorry page
and i have to enter google captcha code.

Cuz it bans me and the program do not work about for 3 hours or more.

How to come around that i can use it my program normaly ?
I'm using regular expressions codes.
And sorry for my bad english.

RE: need a help - ArkPhaze - 03-10-2012

Can you post the code here? My crystal ball seems to be broken so i'm not sure what you're doing wrong. Taking a blind guess though i'm assuming that you don't have any multi-threading and that you're using regular web request methods. Please don't tell me that you use a WebBrowser on top of this too...

RE: need a help - polas - 03-10-2012

No i do not use webbrowser control just regular expressions.
Cuz i hate explorer browser :headbash:

Here is the code

Code:
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        Timer1.Start()
  End Sub

Code:
Private Sub Timer1_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer1.Tick
        Try
         Dim wc As New WebClient
            Dim source As String = wc.DownloadString("http://www.google.com/search?sclient=psy-ab&hl=lt&site=webhp&source=hp&q=" + TextBox1.Text)
            Dim m1 As MatchCollection = Regex.Matches(source, "http(s)?://([\w-]+\.)+[\w-]+(/[\w- ./?%&=]*)?\.(pdf|txt|chm|doc)", RegexOptions.Singleline + RegexOptions.IgnoreCase)
            For Each m As Match In m1
                Dim value As String = m.Groups(0).Value
                ListBox1.Items.Add(value)
            Next
        Catch ex As Exception

        End Try
 End Sub

'remove dups from listbox
        ListBox1.Sorted = True
        ListBox1.Refresh()
        Dim index As Integer
        Dim itemcount As Integer = ListBox1.Items.Count
        If itemcount > 1 Then
            Dim lastitem As String = ListBox1.Items(itemcount - 1)
            For index = itemcount - 2 To 0 Step -1
                If ListBox1.Items(index) = lastitem Then
                    ListBox1.Items.RemoveAt(index)
                Else
                    lastitem = ListBox1.Items(index)
                End If
            Next
        End If
 End Sub

here is i will show you that it works but after 3 hours it stops working
and onther day it works again i tired to clean cookies which google adds as ban and nothing.

Here is the pictute of my program example.

[Image: e93fbbfb28b2dfd47dca967066634d25.png]

[Image: e93fbbfb28b2dfd47dca967066634d25.png]

This program made to grab pdf txt and doc files from any search engine.

You can get anything with this code but come on guys no body knows how to fix the problem ?

Here what can you get more it scrapes and look for php forums if you like
it i have many programs like spider,scraper and so on which i'm working on but this is first problem using regex and do not know a lot of about regex programming

here is the example of the same program.

[Image: 7f25b3b9b3b6e8f9ba6bcd3f763e6695.png]

[Image: 7f25b3b9b3b6e8f9ba6bcd3f763e6695.png]

RE: need a help - polas - 03-11-2012

anyway it is done Smile

but thanks