If this is something you would like to automate, and to implement in C#, you could:
- Use XmlTextReader to process the XML
- Write a stored procedure in MS SQL Server that stores the data coming from XML, with some logic for duplicate data
If this is a one time only process, you could:
- Import the XML data into Excel, and save it as a .csv
- Use SSIS/DTS to import the .csv data into a new table in your database
- Write a query that selects only the non-duplicates from the new table, and inserts them into the table you want.
I could do either approach if you need additional help.