Database

20-Oct-05 11:59

Hi,

I was wondering if there was a better way of reading, updating and writing back text contained in a row in a DataTable in memory.

I need to read the text in a DataRow, and replace one character with another. For example, replace periods with backslashes.

Currently my code does this.

<br />
....<br />
DataTable convertperiodtoslash = new DataTable();<br />
sqlDataAdapter.Fill(convertperiodtoslash);<br />
foreach(DataRow dr in convertperiodtoslash.Rows)<br />
			{<br />
				foreach(DataColumn col in convertperiodtoslash.Columns)<br />
				{<br />
					dr[col] = dr[col].ToString().Replace(".", "\\");<br />
				}<br />
			}

Everything works fine, except that when there are 50,000 rows it takes around 10 mins to process. For smaller number of rows, say around 2000 or so it only takes seconds. So I was wondering if there was a better way of doing this update.. better meaning faster/more efficient code.

I found that the part which slows down the whole process is the writing back into the DataTable. If I write the modified text back into a string array it takes only 5-10 seconds to process even 50,000 rows.

dr[col] = (modified text)
^
|----> bottleneck

Any suggestions/recommendations will be greatly appreciated !!
Thanks in advance!

C# Noob.. Poke tongue | ;-P

S Douglas20-Oct-05 22:52

20-Oct-05 22:52

Have you tried an update query?

UPDATE tbl SET tbl.Col = "\\"
WHERE tbl.Col Like "%.%";

DEBUGGING : Removing the needles from the haystack.

-- modified at 4:57 Friday 21st October, 2005

blackhole07721-Oct-05 9:46

21-Oct-05 9:46

Hi sfdougl ! thanks for responding.

Would, what you suggested work on a DataTable in memory? or how would you do that to a DataTable in memory I guess is my question. I do not want to modify the source (sql) table and be able to update this list and spit it out into an excel file or text file. I also need to display this table in a datagrid to determine extra processing.

Here is my goal..

1. Read sql table..
2. Example row (always a string) "one.two.three"
3. Pull this data from a sql table and store into a DataTable and display in a DataGrid.
4. Output row "one\two\three" by replacing '.' with '\'
5. Also, if need be modify it to "one.two.three.four" - has to be determined at runtime.
6. Output this to a text file. I cannot modify the source.

S Douglas23-Oct-05 22:01

23-Oct-05 22:01

What database technology are you using?
Which technology are you using to access the database?

I know ADO (pre .NET). I might have an option for you but without knowing what you’re using kind of hard to know the answer.

It should be possible to pull the data you seek from the db save it to XML and then modify that to suit the users needs (without looping).

Or you could always create a temp table to modify and drop the table after the data has been modified.

(sorry its taken so long to get back to you, I dont use the computer on the weekends Big Grin | :-D

)

DEBUGGING : Removing the needles from the haystack.

-- modified at 4:15 Monday 24th October, 2005

blackhole07724-Oct-05 9:27

24-Oct-05 9:27

I am using a SQL server backend, and connecting using the SQLClient dataadapter in .NET (Visual Studio .NET 2003). I don't mind creating a temp table to modify, but again I'm unsure of how to bulk modify an entire table without looping. In fact even now I am only using a temp table which i populate using my select query.
I did consider XML, but didn't get the chance to play with it as yet.

thanks again for responding !

S Douglas24-Oct-05 12:29

24-Oct-05 12:29

See my other post about using the Replace command as part of the SQL query. That should do the trick.

DEBUGGING : Removing the needles from the haystack.

S Douglas23-Oct-05 23:40

23-Oct-05 23:40

blackhole077 wrote:
1. Read sql table..
2. Example row (always a string) "one.two.three"
3. Pull this data from a sql table and store into a DataTable and display in a DataGrid.
4. Output row "one\two\three" by replacing '.' with '\'

Simple solution, should work for most RDBMS engines (cant believe I forgot about this one).

If you don’t mind transforming the data when pulling it out of the db use the ‘Replace’ TSQL command. This will not change the underlining data, only its presentation.

SELECT Replace([txt],".","/") AS epr
FROM tbl;

You indicated the display will be determined at runtime. So just do a simple switch or if to determine which way to pull the data. Hows that sound?

Transact-SQL Reference Replace[^]

DEBUGGING : Removing the needles from the haystack.

blackhole07729-Oct-05 18:24

29-Oct-05 18:24

Sorry, it took so long to reply. I will try this out! I also found out that there was more manipulation required with the string than I had anticipated and I'm going in another direction, probably with arraylists or arrays. Thanks for your help! I really appreciate it!

S Douglas30-Oct-05 16:05

Determining whether DataRow values have changed

30-Oct-05 16:05

No problem, always glad to try and help...

DEBUGGING : Removing the needles from the haystack.

GazzaJ20-Oct-05 5:55

GazzaJ

20-Oct-05 5:55

I have an application with a strongly typed Dataset which I populate and then transform the rows in the DataTable into objects used in my application. The application then updates the objects and I call code to update the appropriate DataRows before submitting the updates to the table.

I originally wrote the method to update the DataRow to update each column in the row using the value in the object passed in. This however causes the RowState of the DataRow to change from Unchanged to Modified even if the value has not changed. Therefore I have logic to check the values in the DataRow with the value in the object and only update if there is a change, this logic incorporates checking for null values also.

Is there any method of getting the DataRow to do this automatically?

Someone answer my questions,please!

Deema Jack20-Oct-05 2:55

Deema Jack

20-Oct-05 2:55

Hi every body,
I have two questions which I failed to know their answers:
1- What is the best to make an ADO.NET connection with a DB (in a form in VB.NET) with one table in or with many tables but have a relationship between them through SQL?Why?
2- What is the best technique used in Vb.net to make a form has a connected DB. Is it by writing the code by ourselves Or, by using the available ADO.NET data providers(OLEDB) in the Toolbox in VB.NET?Why?
With Thanks.

-- modified at 8:57 Thursday 20th October, 2005

Re: Someone answer my questions,please!

Colin Angus Mackay20-Oct-05 6:35

Colin Angus Mackay

20-Oct-05 6:35

Deema Jack wrote:
1- What is the best to make an ADO.NET connection with a DB (in a form in VB.NET) with one table in or with many tables but have a relationship between them through SQL?Why?

There are many different ways. All have their pros and cons depending on the situation and what you want to use it for. What do you want to do?

Deema Jack wrote:
2- What is the best technique used in Vb.net to make a form has a connected DB. Is it by writing the code by ourselves Or, by using the available ADO.NET data providers(OLEDB) in the Toolbox in VB.NET?Why?

The best "technique" is to use some sort of data abstraction model. Create a class (or set of classes) whose responsibility is to access the database and let your form get on with the job of dealing with the user interface. Let everything do its thing well, rather than create some sort of monster that cannot do anything well.

If you think this answer is not very good it is because I am attempting to provoke you into starting a discussion where you tell us more about what you are doing and then perhaps we can guide you towards the right way to do it.

My: Blog | Photos

"Man who stand on hill with mouth open will wait long time for roast duck to drop in." -- Confucious

Web part database access

phy6njsb20-Oct-05 0:30

20-Oct-05 0:30

I think I may be missing something simple as I can't find a similiar problem on the internet, so any help would be greatly appreciated. I am trying to access an SQL Server database through sharepoint via a web part. However I have sql client permission errors.

An Error has occurred: Request for the permission of type System.Data.SqlClient.SqlClientPermission, System.Data, Version=1.0.5000.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 failed.

I have tried many different connection strings and authentication modes but always have the same error.

The connection string currently is:"Data Source=lons3003;Initial Catalog=FF;Integrated Security=SSPI;"

Thanks in advance

Michael Potter20-Oct-05 11:23

Michael Potter

20-Oct-05 11:23

Integrated Security? This means you have to give the web service thread the proper rights to access the SQL server. I usually go with a SQL username/password scheme instead of giving IUSR_* anonomous accounts any real domain rights.

phy6njsb20-Oct-05 22:48

20-Oct-05 22:48

Thank you for responding but I still have the same problem using username and password.

server = Server;database=db;uid=sa;persist security info = true;

any other suggestions?

Michael Potter21-Oct-05 3:21

Michael Potter

21-Oct-05 3:21

Your Connection string should look something like this:

"Server=Aron1; Database=pubs; User ID=bob; Password=asdasd; Trusted_Connection=False"

I hope you are not using 'sa' (with or without a password). OMG | :OMG:

I would create a specfic user on the SQL server with the exact rights necessary to accomplish your goals. It doesn't make sense to expose your system to the Internet like that.

You will find a whole set of connection string patterns on: www.connectionstrings.com [^]

phy6njsb23-Oct-05 22:54

23-Oct-05 22:54

Cheers. But I get the same problem using the username and password string.

Michael Potter24-Oct-05 3:15

Michael Potter

24-Oct-05 3:15

Is your web server out of your domain/firewall zone? Maybe you don't have access to talk on the port (default 1433). Try and open the port from a using telnet from a command prompt on the web server (XXX is the IP number of the SQL Server):

telnet XXX.XXX.XXX.XXX 1433

If you go to a blank screen then you are connected (you will have to [X] out of the app at that point). Otherwise a connect error will be displayed.

phy6njsb24-Oct-05 3:53