Click here to Skip to main content
12,074,763 members (61,500 online)
Click here to Skip to main content
Add your own
alternative version

Tagged as

Stats

448.3K views
16 bookmarked
Posted

Import Data from a Text or CSV file into SQL Server

, 26 Nov 2011 CPOL
Rate this:
Please Sign up or sign in to vote.
Import Data from a Text or CSV file into SQL Server

I was recently in need of importing data from a CSV (Comma Separated File) to a table in SQL Server 2008.
After searching in Google for a little while, I found this blog entry from Pinal Dave (SQL Authority) which always provides good content.

Besides passing along the technique described in Pinal's blog post, I would like to expand by explaining a few snags that I ran into.

My source data in the CSV file looks something like this:

HOU009,Windows Phone 7,Will Martinez,
11/10/2011,Houston; TX,999,2
HOU010,WPF for Business Applications,Will Martinez,
11/15/2011,Houston; TX,695,1

More or less, I have all the data needed to fill the columns of my target table. My assumption is that my column id of type uniqueidentifier is going to be auto-generated. This is my table:

CREATE TABLE [dbo].[Courses](
   [id] [uniqueidentifier] NOT NULL DEFAULT NEWSEQUENTIALID(),
   [code] [varchar](50) NOT NULL,
   [description] [varchar](200) NULL,
   [instructor] [varchar](50) NULL,
   [date] [date] NULL,
       [venue] [varchar](50) NULL,
   [price] [money] NULL,
       [duration] [int] NULL,
 CONSTRAINT [PK_Courses] 
    PRIMARY KEY CLUSTERED ([id] ASC,[code] ASC )
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, _
	IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) 
  ON [PRIMARY]) ON [PRIMARY]

As you may notice, I have two datatypes that may be challenging, first my primary key is of type uniqueidentifier and also, I'm using a date datatype.

Initially, I tried to import my data from the CSV file using the below statement:

bulk insert [dbo].[Courses]
from 'C:\Courses.csv'
with (fieldterminator = ',', rowterminator = '\n')
go

On my first try, I did not have any luck. I ran into the below issue:

Msg 4861, Level 16, State 1, Line 1
Cannot bulk load because the file "C:\Courses.csv" 
could not be opened. Operating system error code 5
(Access is denied.).

Moved my file to the Public directory and then made sure permissions were set to 'Everyone'. No luck, although I did not get the "access denied" error. My insert returns 0 row(s) affected.

At this point, my guess was that I had a couple of issues:

  • First, I did not have a value for the Id column which is our primary key
  • My column of type date could be a problem if there is an implicit conversion of string data

To work around the problem, I decided to create a new temporary table, without a primary key and with a datetime datafield. Here is the table:

CREATE TABLE [dbo].[CoursesTemp](
   [code] [varchar](50) NOT NULL,
   [description] [varchar](200) NULL,
   [instructor] [varchar](50) NULL,
   [date] [datetime] NULL,
   [venue] [varchar](50) NULL,
   [price] [money] NULL,
   [duration] [int] NULL)

Ran my insert statement again:

bulk insert [dbo].[CoursesTemp]
from 'C:\Users\Public\Downloads\Courses.csv'
with (fieldterminator = ',', rowterminator = '\n')
go

Success! Finally got to see what I was waiting for...(20 row(s) affected).

At this point, life is easy. I can use the data I just inserted in the temporary table and use it to insert into the "live" table.

insert [dbo].[Courses]
  (code, description, instructor, date, venue, duration)
select 
   code, description, instructor, cast(date as date), venue,
   duration
from [dbo].[CoursesTemp]

Notice that my Id column is not listed since it has a uniqueidentifier with a default of NEWSEQUENTIALID , it automatically generates a GUID for each record.

To deal with the issue of the date field, I noticed that there is a CAST statement that will convert the data to the appropriate datatype.

Below is how the data looks like in my table:

?

Now, I have the data I needed and my id column has nice GUIDs generated for every record.
I'm ready to get some work done.

Hope this helps somebody out there,
Will


<meta name="Import Text file or CSV to SQL Server" />
<meta name="keywords" content="Text File to SQL Server, CSV To SQL Server, C#, T-SQL, Microsoft SQL Server, general Windows 7 issues, Windows 7, Application Admin Mode, Running Stored Procedures WCF Service, iPhone suggestions and recommendations, interesting coding techniques and factoids and personal expertise in writing LINQ statements, working with ListViews, Content Panels, methods, functions and other general software development topics, ASP.NET, C#, Microsoft SQL Server 2008, Microsoft SQL Server 2008, SSRS, SSRS 2008, LINQ, Windows 7, iPhone, ListViews, Webparts, Content, Panels, Stored procedures, SP, T-SQL, SELECT, UPDATE, INSERT, DELETE, XML, App.Config, Web.Config, Application Settings, HTML, Javascript, Exporting XML Content StreamWriter, DotNetZip Example"/>
<meta name="robots" content="index, follow" />

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

gmtzgtz
United States United States
No Biography provided

You may also be interested in...

Comments and Discussions

 
QuestionIs there another way to do this, but without Bulk Insert? Pin
Boris Krepkanovich15-Feb-15 0:07
memberBoris Krepkanovich15-Feb-15 0:07 
QuestionCSV not supported by SQL Server Bulk import Pin
nanonerd18-Sep-13 11:34
membernanonerd18-Sep-13 11:34 
QuestionQuestion about import txt file.. please help Pin
GaboBigale16-Jan-13 6:35
memberGaboBigale16-Jan-13 6:35 
Hey, great article can you help men with this… i had read a txt file with the bulk sintax but after that i find another problem, in my txt file it has the same header and detail in it so my problem now is how i can to put the header in a table1 and the detail in a table2, another thing i create just a field1 for put all my rows(header and detail), but they have FIELDTERMINATOR = ‘|’, but the header it has seven fields and the detail it has eleven fields so thats another problem because i cannot create just one table with the exact fields to put my detail and header… this an example of a txt file:

HR|001580/06|RG|11/01/2013 12:00|BG|3573|001580 IT|001580/01|1|00147066||1200|852.3|830.3|1.35|UNIDAD|0|31/12/2014 00:00 IT|001580/02|1|00147066||200|852.3|830.3|1.35|UNIDAD|1|31/12/2014 00:00 IT|001580/03|1|00147066||100|852.3|830.3|1.35|UNIDAD|55|31/12/2014 00:00 IT|001580/04|2|00254276||200|852.3|830.3|1.35|UNIDAD|0|31/12/2014 00:00 IT|001580/05|3|00305359||1700|852.3|830.3|1.35|UNIDAD|0|31/12/2014 00:00 IT|001580/06|3|00305359||300|852.3|830.3|1.35|UNIDAD|1|31/12/2014 00:00

and this is the query i have…

CREATE TABLE #temp
(
campo1 VARCHAR(max)
)
BULK INSERT #temp
FROMWITH
(
FIELDTERMINATOR = ‘|’,
ROWTERMINATOR = ‘\n’
)
This firs query it is for the header

SELECT *
FROM #temp
WHERE SUBSTRING(campo1,1,2) = ‘HR’
This firs query it is for the detail

SELECT *
FROM #temp
WHERE SUBSTRING(campo1,1,2) = ‘IT’
so then come my problem how i can Separate the fields for the header and the detail.
AnswerRe: Question about import txt file.. please help Pin
gmtzgtz16-Jan-13 7:46
membergmtzgtz16-Jan-13 7:46 
GeneralRe: Question about import txt file.. please help Pin
GaboBigale17-Jan-13 7:22
memberGaboBigale17-Jan-13 7:22 
General:) Pin
Hector Perales26-Apr-12 21:05
memberHector Perales26-Apr-12 21:05 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.160208.1 | Last Updated 26 Nov 2011
Article Copyright 2011 by gmtzgtz
Everything else Copyright © CodeProject, 1999-2016
Layout: fixed | fluid