Click here to Skip to main content
Licence 
First Posted 24 Jan 2006
Views 47,284
Bookmarked 30 times

Batch Update on a very huge table

Updating a huge table in Production , without locking the whole table .
 
Part of The SQL Zone sponsored by
See Also
2 votes, 50.0%
1

2
1 vote, 25.0%
3

4
1 vote, 25.0%
5
2.85/5 - 4 votes
μ 2.85, σa 3.35 [?]

Those of us who work in the Production / Live DB , know the pain , of firing a update query on a huge table . This is very problematic as this will lead to the following

  • Lock escalation : The entire table will be locked by your application , so other users /  application will not be able to perform the DDL on this table
  • The TempDB will grow huge
  • A update lock on a table will stop other users , for making changes in the table

I have written a below snippet which will help us fight the above issues

-- Step 1 : Declare the varaibles

use DBNAME
Declare @counter int
Declare @RowsEffected int
Declare @RowsCnt int
Declare @CodeId int
Declare @Err int
SELECT @COUNTER = 1
SELECT @RowsEffected = 0

 


/*
Step 2 : Get the value of the Code , with which we need to update the existing Code . In my case I am capturing is from a table , we can always hard code it .
*/


SELECT @CodeID = CodeID FROM CODE WHERE XXXX ='YYYY'


/*
Step 3: Start the while loop ,  if we have 100,000 records , and in each loop 5,000 records will be update , so the total number of cycle will be 100,000/5000 i.e 20
*/

WHILE ( @COUNTER > 0)
BEGIN
SET ROWCOUNT 5000


-- Note : The SET ROWCOUNT 5000   will just pick the top 5000 records */

   
      /*UPDATING TABLE */
  UPDATE Table
SET CodeID= @CodeID
WHERE Codeid = @OldCode
SELECT @RowsCnt = @@ROWCOUNT ,@Err = @@error
IF @Err <> 0
BEGIN
Print 'Problem Updating the records'
END
IF @RowsCnt = 0
SELECT @COUNTER = 0
ELSE

/* Increment the Counter */

  SELECT @RowsEffected = @RowsEffected + @RowsCnt
PRINT 'The total number of rows effected :'+convert(varchar,@RowsEffected)

            /*delaying the Loop for 10 secs , so that Update is comepleted*/    

     WAITFOR DELAY '00:00:10'         
 END


--Step 4 : Check if all the records are updated or not .
IF EXISTS ( SELECT CodeID
FROM Table (NOLOCK)
WHERE CodeID = @OldCodeid
)
BEGIN
PRINT ('All the records are not updated , there is some problem , Contact Devs ')
END
BEGIN
PRINT ('All the records are updated SUCCESSFULLY !!!!')
END
/* ------Set rowcount to default ----*/
SET ROWCOUNT 0

 

 

 

 

 

 


 

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

About the Author

Himanshu Kumar Sinha

Web Developer

United States United States

Member
He is working with reputed IT company in Hyderabad India .
 
He has got an total IT expereince of 4 yrs . I have been working with VB , .NET & SQL SERVER 2000 & Yukon .
 
he has done his B.Tech In Computer Science & Engg from North Eastern Regional Institute Of Science & Technolgy ( NERIST ) Arunachal Pradesh , Indian .
 
He enjoy's working on the latest technology , driving & cooking .
 


Sign Up to vote   Poor Excellent
Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

 
You must Sign In to use this message board. (secure sign-in)
 
Search this forum  
 FAQ
    Noise  Layout  Per page   
  Refresh
GeneralDynamic Parallelization without Coding PinmemberMember 79040182:13 7 May '11  
GeneralClumsy Implementation PinPopularmemberncarey11:26 1 Feb '06  
For starters, the WAITFOR statement is completely unnecessary. Each statement in a batch executes synchronously. Under SQL Server's default "autocommit" mode (SET IMPLICIT_TRANSACTION OFF), each statement is atomic. Once the UPDATE statement has completed, the transaction has been COMMITed and all locks have been released.
 
A simpler implementation is something like the stored procedure below.The [potential] drawbacks to this technique are twofold:
 
1. An additional UPDATE is peformed that will update no rows (that is, if your batch size is 1000 and you have 5000 rows to update, 6 UPDATE statements will be executed: 5 will affect 1000 rows and the 6th will affect zero (0) rows. Depending on the size of your table, the semantics of your UPDATE and its execution plan, and other work going on in SQL Server, the additional work entailed by "doing it in chunks" might actually result in worse performance than doing it in a single large UPDATE. Consult your DBA (who probably has a better handle on system load and lock contention than you do) to see if this sort of optimization is necessary.)
 
2. This sort of optimization breaks the ACID (Atomicity, Consistency, Isolation and Durability) property of a relational database. You may intend to alter 1000 rows of a table. If you do it in a single transaction, the change will either occur or not occur. If you break it up into multiple transactions by chunking it like this, the state of the database may change mid-stream — other users may alter/delete/add data before you get to it, etc. Whether or not this is an issue, depends of course, a lot of different (and external) factors that can't be addressed in a general way: the exact semantics of your application and your data model for starters.
 
Anyway, here's a cleaner implementation. Note that the stored procedure could also, for instance, retrieve the 'batch size' from a configuration table (rather than receiving it as a parameter). This would allow the DBAs to tune the batch size as needed without required code changes or stored procedure changes.
 
dbo.sp_MassiveUpdate -----------------------------------------------
--
-- A stored procedure to do a big update against a table in small chunks
-- so as to minimize lock contention and transaction length.
--
------------------------------------------------------------------------
create procedure dbo.sp_MassiveUpdate
 
  @old_value  int ,      -- old value
  @new_value  int ,      -- new value
  @batch_size int = null -- optional. If omitted or specified as 0 or NULL,
                         --   the update will be done as a single, large transaction.
 
as
 
  --
  -- local variables
  --
  declare
    @msg            varchar(4000) ,
    @rows_processed int
 
  --
  -- standard SET options
  --
  set nocount                 on  -- eliminates extraneous network round-trips.
  set ansi_nulls              on  -- expect standard ANSI null behaviour
  set concat_null_yields_null on  -- expect standard ANSI null behaviour
  set xact_abort              on  -- rollback and bail on error
  set implicit_transactions   off -- ensure we're running under the default (autocommit) mode
 
  --
  -- put optional argument into canonical form
  --
  set @batch_size = case coalesce(@batch_size,0) when 0 then 0 else @batch_size end
 
  --
  -- if small batches were asked for and we're already in a transaction, hurl
  --
  if ( @batch_size <> 0 and @@TRANCOUNT > 0 )
    begin
 
      set @msg = 'ERROR: '
               + 'Can''t Update in Batches. '
               + 'This stored procedure is running within an uncommitted transaction.'
 
      RAISERROR( @msg , 16 , 1 )
 
      return -1
 
    end
 
  --
  -- do the massize update in batches, if requested. If @batch_size is 0,
  -- then the first batch will be the only batch ( SET ROWCOUNT 0 says that
  -- there are no limits on the number of rows to process
  --
  set @rows_processed = -1
 
  while ( @rows_processed <> 0 )
    begin
 
      set rowcount @batch_size         -- set the batch size
 
      update my_really_large_table     -- perform an update
      set   value_column = @new_value
      where value_column = @old_value
 
      set @rows_processed = @@ROWCOUNT -- get the number of rows affected
 
      set rowcount 0                   -- restore the default batch size
 
    end
 
  --
  -- return to the caller with the return code set.
  --
  return 0
 
GO

GeneralSuggestion: Before & After PinmemberDrew Noakes23:14 30 Jan '06  
GeneralErrors Pinmembernikumi7:01 30 Jan '06  
GeneralMore Details PinmemberJohn Kendrick8:11 24 Jan '06  
GeneralRe: More Details Pinmembertoticow23:34 24 Jan '06  

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Mobile
Web02 | 2.5.120210.1 | Last Updated 24 Jan 2006
Article Copyright 2006 by Himanshu Kumar Sinha
Everything else Copyright © CodeProject, 1999-2012
Terms of Use
Layout: fixed | fluid