How to remove duplicate columns from the csv file using C#

Question

1.00/5 (1 vote)

See more:

I have 2 csv file i need to merge in one file and need to remove duplicate column in merged csv file

What I have tried:

var first = File.ReadAllLines("C:\\Task\\Input\\UserInfo.csv");
            var second = File.ReadAllLines("C:\\Task\\Input\\PersonalInfo.csv");
            var result = first.Zip(second, (f, s) => string.Join(",", f, s));
            File.WriteAllLines("C:\\Divyashree\\Dot_net\\Task\\Input\\combined.csv", result);

Posted 29-Apr-21 21:39pm

Member 15088142

Updated 29-Apr-21 22:14pm

Add a Solution

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

OriginalGriff · Answer 1 · 2021-04-29T22:14:00

In order to do that, you first have to read your file as a CSV - and find some way to relate "row X" of file 1 with "row Y" of file 2 - then identify the duplicate columns to build a new composite dataset which you can write to a new file.

Just reading lines won't do it: that's treating CSV data as if it was a "straight text file" which it is, but that text has a layer of formatting applied which you need to utilize in order to get this to work.

And ... a CSV data column can be spread across multiple lines, if inclosed in double quotes - so you current approach will not only not allow you to identify any columns, but could seriously mess up your data integrity if you are unlucky.

I'd start by reading each CSV file into a DataTable (A Fast CSV Reader[^] could help you there) then combining them into a single new DataTable to write out to a new file.
It's a lot less effort (and more maintainable) than trying to "roll your own" CSV data reader!