What is the best way to Merging two datasets in c#?

Question

1.00/5 (1 vote)

See more:

dataset1 will have a collection of awards and the dataset2 will a collection vendors. single award can be related to multiple vendors. so the final merged dataset should looks like the following.

dataset1.record1+dataset2.record1
dataset1.record1+dataset2.record2
dataset1.record1+dataset2.record3
dataset1.record1+dataset2.record4
.....
.
.
.
.
dataset1.record1+dataset2.recordN

dataset1.record2+dataset2.record1
dataset1.record2+dataset2.record2
dataset1.record2+dataset2.record3
dataset1.record2+dataset2.record4
.....
.
.
.
.
dataset1.record2+dataset2.recordN and so on... I have tried this with LINQ it is working but it is taking long time to merge. It takes ~4 min to merge 1000 awards with 3500 vendors. Is there any other method to improve the performance?

Posted 4-Nov-14 23:24pm

Dhinesh Kumar M

Updated 9-Nov-14 18:01pm

v2

Add a Solution

Comments

syed shanu 5-Nov-14 5:39am

Why do you want like this.You can also result as one dataset by joining both table and produce one select statement

V. 5-Nov-14 5:56am

Searched for "msdn dataset merge" in google and the first hit gave me this: http://msdn.microsoft.com/en-us/library/system.data.dataset.merge(v=vs.110).aspx That would be the first thing to investigate.

Vijay Gill 5-Nov-14 8:46am

My money will be on a dataset which has these two tables and a proper relationship setup among them. Then just fetch the two tables with parent first and child later. This way you bring back the minimum amount of data from server, and use minimum memory because there is no redundant data!

BillWoodruff 6-Nov-14 5:44am

Good answer !

Maciej Los 6-Nov-14 12:52pm

What kind of database: SQL server, MS Access, MySQL, PostgreSQL, etc?

Dhinesh Kumar M 9-Nov-14 23:52pm

@Maciej, it's MS SQL database.

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Maciej Los · Accepted Answer · 2014-11-06T06:59:00

Solution 2

There are at least 2 methods:
1) using SQL[^]

SQL

SELECT <Field_List>
FROM Awards AS A <INNER|LEFT|RIGHT|CROSS> JOIN Vendors AS V ON A.PrimaryKey = V.ForeignKey

Have a look here: Visual Representation of SQL Joins[^]

2) using Linq[^]
When the datasets have been filled out, you can use Linq queries to fetch data from them.
See this: Method-Based Query Syntax Examples: Join (LINQ to DataSet)[^]

Posted 6-Nov-14 6:59am

Maciej Los

Updated 6-Nov-14 9:36am

v2

Comments

Dhinesh Kumar M 9-Nov-14 23:58pm

Thanks Maciej, I've tried these approaches and its working, but the performance is very low.
It takes ~4 mins to merge 1000x3500 rows and 49 columns.

Maciej Los 10-Nov-14 3:41am

Did you set indexes? There is tons of reasons of above behaviour. On my server joining much, much bigger tables is less time consuming (about ~10-30sec.).