|
Hi,
I am Working on the VB.Net to perform a Multithreading to correct the invalid email addresses in Database. The problem is that I have 22,00,00,000 email addresses. So I want, Multiple threads do work on this process .
I am Little bit confusion about how to create more threads and all the threads can work togather but not access the same IDs of Email Address from table .
For doing this I ahve created a list box , in which I devided the IDs of email Addressses into many list items . I want that first thread do work on first list item , Second created thread do work on second list item and so on.
But the problem is that when i created multithreading some list of item is missing and the threads cannot do work on some list of items.
With Regards,
|
|
|
|
|
Multithreading will not actually make your application perform multiple updates at the same time. Instead it will just have the one thread perform one, then the second one, then the third one, then the first one again, etc.
It will be much faster doing the whole thing with SQL. You wouldn't even need an application at all, just a stored procedure or an SQL job.
My advice is free, and you may get what you paid for.
|
|
|
|
|
I think you do not understand the problems exactly.
I ahve lots of email addresses in my database .
and a single query take a lot of time of its execution.
so in this satuation it will happns that may the whole prrocess take
few days in completing which is not useful for me.
thnks!
with best wishes!!!!!
|
|
|
|
|
I do understand what you mean. What I am trying to say, is that any application using threading, you write, will always use the capacity of your PC or server in a less efficient way than the database server program would.
Again, multithreading in principle does not make the computer perform multiple actions at the same time. It only makes the computer perform these actions in rapid succession, so it looks like it is doing things at the same time.
Your database server program however can and will (usually) readily use multicore CPU's or servers with multiple CPU's. So it will actually make your PC or server perform multiple actions at the same time.
If you need to sort through that much data, you are going to have to be prepared for a lengthy process, although I seriously doubt it would take days. Hours maybe.
So it would be best if you create one additional field in your table, where you can flag each email address when it has been checked.
That way you can write your SQL so that it only checks unflagged addresses, and your sql job or stored procedure will only have to run for a long time the first time.
My advice is free, and you may get what you paid for.
|
|
|
|
|
But, as Johan quite rightly said, 2 threads wont perform the process twice as quick as 1 thread. Threading just means that you can have more than 1 process running independently of each other, but they still have to share processor time.
EDIT : Actually, I could be wrong here, maybe multi-core processors can run threads concurrently? Maybe someone can shed a light on this?
modified on Friday, April 17, 2009 8:15 AM
|
|
|
|
|
Liqz wrote: Actually, I could be wrong here, maybe multi-core processors can run threads concurrently?
If they couldn't, what would be the point of multiple cores??
|
|
|
|
|
I realise that processes can be run concurrently using multi-core processors but wasn't sure about threads.
Anyway, thanks for correcting my mistake 
|
|
|
|
|
Processors don't execute processes, they execute threads. A process is an operating system concept, not a processor concept. A process is just an organized collection of threads.
|
|
|
|
|
Ah I see, thanks again
|
|
|
|
|
You're not getting it. The SQL Server is a better place to do this. You won't be wasting a ton of time transferring that much data TWICE acrossed the network to the PC doing the processing, plus, multi-threading will only buy you a performance benefit if the PC has multiple processors, or multiple cores. You can't start, say, 10 threads on a single core machine and expect it to go 10 times faster. It doesn't work that way. Starting more threads to process data than there are cores in the machine is a complete waste of time. In a single core machine, the processor can only execute one thread at a time, pausing all other threads until it can get around to each of them.
|
|
|
|
|
220 MILLION or 22 BILLION?
Thats a lot of email addresses. Sort of like a list a spammer would have.
Either way, its going to take a LONG time to process that lot.
|
|
|
|
|
Given either number, I'm not helping this goof ball send spam. I purging my hotmail acct now - what a pain in the butt.
Any suggestions, ideas, or 'constructive criticism' are always welcome.
"There's no such thing as a stupid question, only stupid people." - Mr. Garrison
|
|
|
|
|
Hello everyone,
I have a datagridview and want to insert each row of this into a database(db1) table(sheet1). Both the datagridview and sheet1 has 7 fields. I used the following code but I just wonder what I should fill into the values for insert into statement, provided that except for field1 all of the data is in text format. Field1 is in number format:
Dim dgr1 As DataGridViewRow
For Each dgr1 In Me.DataGridView1.Rows
Dim con As OleDbConnection = New _ OleDbConnection("Provider=Microsoft.jet.oledb.4.0;data source=|DataDirectory|\db1.mdb")
con.Open()
Dim cmd As OleDbCommand = New OleDbCommand("INSERT INTO Sheet1(Field1, _ Field2, Field3, Field4, Field5, Field6, Field7) values()", con)
cmd.ExecuteNonQuery()
con.Close()
Next
Thanks in advance.
|
|
|
|
|
If the fields in your database are all TEXT, CHAR, VARCHAR, OR NVARCHAR (i.e. strings) then you can feed pretty much whatever you want into those fields. The fields will accept text and numerical values. Only your first field will require a numerical value. The only thing you need to check is the maximum length of each field against the possible maximum length of each value that you might feed to it.
My advice is free, and you may get what you paid for.
|
|
|
|
|
many thanks for the suggestions.
Can you please give me an example code to fill into value?
|
|
|
|
|
Well the question is what kind of inserting you want to do.
Simply put you could do something like this:
Dim cmd As OleDbCommand = New OleDbCommand("INSERT INTO Sheet1(Field1, Field2, Field3, Field4, Field5, Field6, Field7) values(1,2,3,4,5,6,7)", con)
or
Dim cmd As OleDbCommand = New OleDbCommand("INSERT INTO Sheet1(Field1, Field2, Field3, Field4, Field5, Field6, Field7) values(1," & SomeVariable & "," & SomeOtherVariable & "," & YetAnotherVariable & ",5,6,7)", con)
or
Dim cmd As OleDbCommand = New OleDbCommand("INSERT INTO Sheet1(Field1, Field2, Field3, Field4, Field5, Field6, Field7) SELECT field1, field2, field3, field4, field5, field6, field7 FROM AnotherTable WHERE some condition is true ", con)
My advice is free, and you may get what you paid for.
|
|
|
|
|
Since I am looping through each row of a datagridview, I would like to insert each row of the datagridview. What should I do in this case?
Many Thanks
|
|
|
|
|
Use the second option from my previous answer, and set the value of each variable to the value you get from each field in each row in the datagridview
Something like:
Variable1 = row.Item("Field1")
Variable2 = row.Item("Field2")
etc.
My advice is free, and you may get what you paid for.
|
|
|
|
|
I tried:
"INSERT INTO Sheet1(Field1, Field2, Field3, Field4, Field5, Field6, Field7) Values(" & dgr1.Cells(0) & ", " & dgr1.Cells(1) & ", " & dgr1.Cells(2) & ", " & dgr1.Cells(3) & ", " & dgr1.Cells(4) & ", " & dgr1.Cells(5) & ", " & dgr1.Cells(6) & ", " & dgr1.Cells(7) & " )"
but it says syntax error
|
|
|
|
|
Something like that, yes.
Also adapt your code like this, so you will know if something goes wrong:
Try
'Your code here
Catch ex As Exception
MsgBox(ex.ToString)
End Try
and remember to check the length of the content of each cell against the maximum length of each field.
My advice is free, and you may get what you paid for.
|
|
|
|
|
When I did like this:
Try
Dim dgr1 As DataGridViewRow
For Each dgr1 In Me.DataGridView1.Rows
Dim con As OleDbConnection = New OleDbConnection("Provider=Microsoft.jet.oledb.4.0;data source=|DataDirectory|\db1.mdb")
Dim cmd As OleDbCommand = New OleDbCommand("INSERT INTO Sheet1(Field1, Field2, Field3, Field4, Field5, Field6, Field7) Values(" & dgr1.Cells(0) & ", " & dgr1.Cells(1) & ", " & dgr1.Cells(2) & ", " & dgr1.Cells(3) & ", " & dgr1.Cells(4) & ", " & dgr1.Cells(5) & ", " & dgr1.Cells(6) & ")", con)
con.Open()
cmd.ExecuteNonQuery()
con.Close()
Next
Catch ex As Exception
MsgBox(ex.ToString)
End Try
It said:
Error 1 Operator '&' is not defined for types 'String' and 'System.Windows.Forms.DataGridViewCell'.
I don't know what values I should enter. This must be a simple stuff but I am just struck.
|
|
|
|
|
Yep, that's why the best first step in learning this stuff is the Try Catch statement.
okay, so don't panic, this problem is something that you are going to bump into a lot when coding, and the solution is very simple:
for example dgr1.Cells(0) is in fact the cell as an object, not the cell's content. So all you need to do is get to the object's (your datagridview cell) content.
try dgr1.Cells(0).Value, and see what it says next.
My advice is free, and you may get what you paid for.
|
|
|
|
|
Thank you very very much. Following code works:
Try
Dim dgr1 As DataGridViewRow
For Each dgr1 In Me.DataGridView1.Rows
Dim con As OleDbConnection = New OleDbConnection("Provider=Microsoft.jet.oledb.4.0;data source=|DataDirectory|\db1.mdb")
Dim cmd As OleDbCommand = New OleDbCommand("INSERT INTO Sheet1(Field1, Field2, Field3, Field4, Field5, Field6, Field7) Values('" & dgr1.Cells(0).Value & "', '" & dgr1.Cells(1).Value & "', '" & dgr1.Cells(2).Value & "', '" & dgr1.Cells(3).Value & "', '" & dgr1.Cells(4).Value & "', '" & dgr1.Cells(5).Value & "', '" & dgr1.Cells(6).Value & "')", con)
con.Open()
cmd.ExecuteNonQuery()
con.Close()
Next
Catch ex As Exception
MsgBox(ex.ToString)
End Try
But I just wonder why I am getting one extra blank row in my database table.
|
|
|
|
|
That is because the datagridview offers an empty line at the bottom for the user to enter new rows.
You can prevent this row from being entered into your database by checking if all cells are empty or not.
So instead of putting dgr1.Cells(0).Value directly into the insert statement, you could do something like this:
If Not dgr1.Cells(0).Value & dgr1.Cells(1).Value & dgr1.Cells(2).Value & etc. = "" Then
Dim cmd As OleDbCommand = New OleDbCommand("INSERT INTO Sheet1(Field1, Field2, Field3, Field4, Field5, Field6, Field7) Values('" & dgr1.Cells(0).Value & "', '" & dgr1.Cells(1).Value & "', '" & dgr1.Cells(2).Value & "', '" & dgr1.Cells(3).Value & "', '" & dgr1.Cells(4).Value & "', '" & dgr1.Cells(5).Value & "', '" & dgr1.Cells(6).Value & "')", con)
End If
My advice is free, and you may get what you paid for.
|
|
|
|
|
But I am getting two blank rows. One is the obvious one but I am getting one another row on the top of the default blank row.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
