|
Why are you using a Queue?? A queue is a list of items that's organized and manipulated on a First-In, First-Out basis.
What's the purpose of this collection?? The type of collection you use isn't so much dictated by what it's going to hold, but how the collection is going to be used.
If you want to manipulate items, adding them and removing them, then a List(Of T) would work, an array of strings, ArrayList, ...
|
|
|
|
|
Actually I'm working on a Distributed Web Crawler a Multithreaded application where each URL discovered will going to be added in a queue to implement the breadth-first search model. Many threads will access this queue and get the URLs from the top and then add the newly discovered URLs at the end of the queue and them remove the carwled URLs from the queue. Now what should i use in such a scenario. but when i implemented this logic i get this error "Collection was modified; enumeration operation may not execute".
AliAmjad (MCP)
|
|
|
|
|
Why are you enumerating them?? I don't see anything in this description that would justify it.
Well, now I see where your code came from. It's a modification of the sample code on MSDN.
You don't need a loop to dequeue items. The Dequeue method pops the top item off the queue and returns it an as Object. You have to cast the Object back to a String in order to use it.
Dim url As String = DirectCast(que.Dequeue(), String)
|
|
|
|
|
OKKK i get it but as i am in a multithreaded application then lots of threads will access this queue then should i use SyncLock or what else to ensure thread safety.
AliAmjad (MCP)
|
|
|
|
|
Wrap the queue object in a class that implements adding and dequeuing. Use the SyncLock on an object to make it thread safe.
|
|
|
|
|
Cooool thanks buddy but I'm going to save MD5 hash algorithm of URLs in hexadecimal form in the Queue so which Queue(of T) i should use so that It'll consume as little memory as possible because I am gonna Enqueue it with lots of URLs. Again thanks for your valuable time Dave.
AliAmjad (MCP)
|
|
|
|
|
AliAmjad wrote: but I'm going to save MD5 hash algorithm of URLs in hexadecimal form in the Queue so which Queue(of T) i should use so that It'll consume as little memory as possible because I am gonna Enqueue it with lots of URLs
That's nice. You better do some research on this first. MD5 is a one-way encryption hash. You can't reverse the hash and get the original URL back.
Forget hashing or compressing the URL. It's adding more complexity than you need, and unjustifiably so.
|
|
|
|
|
Yes it is a one-way encryption hash and that's why I'll use these for the reference of original URLs saved in a file in this way I'll be able to store many URLs in memory because a web crawler needs to maintain a huge list of URLs in memory and also i'll calculate the MD5 for the page itself for preventing the duplicate entries in index because its size is very small and always equal. So what you suggest on this. Should i go with MD5 or just manipulate raw URLs in memory but this way it will consume lot of memory (RAM).
AliAmjad (MCP)
|
|
|
|
|
I don't see the need for MD5 anything in your app. On top of that, with the skill level you've shown in the original post, this shouldn't even be a concern to you right now. Just getting the basic functionality should be. Queue up URL's, dequeue one, download it and move on. THEN you can add complex indexing and hashing to your database.
|
|
|
|
|
Thank you very much man for helping me out I'll surely implement this valuable Logic and analyze the results then I'll tell you about it.
Dave Kreskowiak wrote: The skill level you've shown in the original post
I consider myself as a beginner because I always try to grasp these important concepts and I've learned allot from you. What do you think about my Skill level? Am i capable enough to take this challenge of making a Distributed Web Crawler although I am up to some degree successfully able to achieve it. But still need a lot of guidance and help form people like you.
AliAmjad (MCP)
|
|
|
|
|
I want to access a file bit by bit.
My problem is that .Net as not "bit" datatype other than boolean.
It has bitarray which could be fine, I'm not sure.
I already managed to read the file by looping through all its bytes by using the ReadAllBytes method but now I want to be able to alter the bits and write them back to a file. This file could be anything, a good example is a JPG image.
How should I go about doing this?
|
|
|
|
|
That's where using masking, bitwise and logical operators, and math comes into play. If you're doing image processing, I highly suggest reading Christian Graus articles on "image processing for dummies[^]".
THe code is in C# because VB.NET doesn't support pointers.
|
|
|
|
|
I have two XSD File. one is master and second is its child. i m creating the dataset from child XSD. now i m setting the data into dataset and then generating xml file from this dataset. it is working fine. But the problem is that the sequence(means order of the fields defined in Master Xsd File) defined in master file is not following in the xml file created thru DATASet.
can any body help me.
kishore
-- modified at 1:02 Saturday 13th October, 2007
|
|
|
|
|
kkb_2001 wrote: But the problem is that the sequence defined in master file is not following in the xml file created thru DATASet.
Sequence?? What "sequence"?? What are you talking about??
|
|
|
|
|
could anyone explain to me the codes dat are written for the dataset and datareader.............
-- modified at 23:39 Sunday 14th October, 2007
|
|
|
|
|
Could you please provide a permalink to "Adams post"?
|
|
|
|
|
Guys im having a problem with this statement..
ALTER TABLE tblSample
ADD SampleField nvarchar (30) NOT NULL
CONSTRAINT SampleField PRIMARY KEY
im just trying to set my newly added column and at the same time setting it as a primary key.. but it wont work..
Is there any other way?
or can it set an existing non primary column to a primary one.. if it can then how.. thanks..
|
|
|
|
|
This should work. What error are you getting?
BTW, you should really ask this question in the SQL forum. It has nothing to do with VB.
-- modified at 8:16 Friday 12th October, 2007
Paul Marfleet
|
|
|
|
|
it will not work because in sql u can not add column with Null having a primary key. it will work in access after removing nvarchar->char
-- modified at 4:34 Saturday 13th October, 2007
|
|
|
|
|
I'm not sure why you replied to my message instead of the original poster. Anyway, it's pretty obvious from the query syntax that the database being used is SQL Server not Access.
Paul Marfleet
|
|
|
|
|
|
Or just delete the question... that works too. 
---
single minded; short sighted; long gone;
|
|
|
|
|
What was the deleted question?
"Any sort of work in VB6 is bound to provide several WTF moments." - Christian Graus
|
|
|
|
|
Hi friends..
I have 3 tables in my dataset..I need to bind them to a single data grid.All the tables in the dataset have varying fields..How could i do this..Someone help in this regard as early as possible..
Regards,
Balaguru
|
|
|
|
|
hi
Use joins, and Add relations, if you are going to update the data later. you have to use DAl(Data Access Layer)
Hope it helps
Vuyiswa Maseko,
Sorrow is Better than Laughter, it may Sudden your Face, but It sharpens your Understanding
VB.NET/SQL7/2000/2005
http://vuyiswamb.007ihost.com
http://Ecadre.007ihost.com
vuyiswam@tshwane.gov.za
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
