Click here to Skip to main content
15,891,607 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
We have 2 crore data and we need 3 columns in select based on where condition using like
It takes lot of time.
I have used full text search but its limitation we can apply on only one columb
One column * one index
What i can do to make it fast
Any idea?

What I have tried:

I have used full text search but its limitation we can apply on only one colum
Posted
Updated 10-Jan-17 0:22am
Comments
Wendelius 9-Jan-17 23:54pm    
Please post the whole query in order to see the query structure.
Daniel Jones 10-Jan-17 0:05am    
SELECT DISTINCT TOP (@SearchLimit) fl_st_id AS [Key],
LTRIM(RTRIM(fl_st_id))+' - '+ fl_st_name AS [Description] FROM FCLEGS WHERE fl_st_id like @SearchTer
Patrice T 10-Jan-17 6:25am    
Use Improve question to update your question.
So that everyone can pay attention to this information.
Daniel Jones 10-Jan-17 0:07am    
fl_st_id nonclustered located on PRIMARY

1 solution

It took quite some time to make a relevant example with larger amounts of data but here it goes...

The first thing is that you can index multiple columns in a single table using full text index.

Another thing is that when using full text indices, use CONTAINS function.

And third thing is that the amount of rows satisfying the condition should be relevantly small compared to the overall amount of rows.

Consider the following example:

Create the test table
SQL
CREATE TABLE FullTextTest (
  id int identity(1,1) NOT NULL,
  textcol1 varchar(max) NOT NULL,
  textcol2 varchar(max) NOT NULL
);

ALTER TABLE FullTextTest 
ADD CONSTRAINT pk_FullTextTest 
PRIMARY KEY (id);

Add a large set of rows
SQL
INSERT INTO FullTextTest (textcol1, textcol2)
SELECT 
   'Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam at elit enim. Mauris et sapien vitae nulla convallis maximus non sit amet nunc. Sed quis consequat eros, nec suscipit nulla. Suspendisse potenti. Duis a posuere dui. Donec sollicitudin eget enim sit amet hendrerit. Nullam neque nisl, sodales ac magna suscipit, iaculis interdum lectus. Integer vehicula egestas augue, finibus commodo urna feugiat et. Integer cursus, lorem ut convallis fringilla, dui elit posuere turpis, nec porta erat ligula quis tortor. Mauris imperdiet ipsum diam, sed eleifend elit blandit sit amet. Fusce eu porttitor elit. Cras tempor orci quis vestibulum tincidunt. Maecenas auctor metus eget diam pulvinar, eget commodo augue pellentesque. Integer congue vitae tellus ut sollicitudin. Nulla non augue ligula.' as data1,
   'Nulla ullamcorper, justo quis malesuada imperdiet, arcu metus consectetur diam, quis consectetur arcu tellus nec magna. Ut faucibus risus ut ex placerat, et scelerisque dui vehicula. Nullam in aliquet lorem. Quisque lobortis quam at dui bibendum gravida. Ut in sodales sapien, quis pretium neque. Mauris feugiat urna dolor, sit amet fringilla mi accumsan sit amet. Vivamus sit amet dictum quam, vel condimentum lorem. Cras elementum eu massa lobortis pharetra. Quisque fringilla volutpat mauris, sed feugiat velit accumsan sed. Nam venenatis nec justo eget consequat. Donec vitae lorem hendrerit nulla pulvinar eleifend. Curabitur ut odio posuere ipsum faucibus commodo sed a est. Quisque tincidunt augue nec mollis volutpat. Maecenas elementum tempus velit vitae condimentum. Integer fermentum mi sapien.' as data2
FROM sysobjects s1, sysobjects s2, sysobjects s3;

INSERT INTO FullTextTest (textcol1, textcol2)
SELECT 
   'Nulla ullamcorper, justo quis malesuada imperdiet, arcu metus consectetur diam, quis consectetur arcu tellus nec magna. Ut faucibus risus ut ex placerat, et scelerisque dui vehicula. Nullam in aliquet lorem. Quisque lobortis quam at dui bibendum gravida. Ut in sodales sapien, quis pretium neque. Mauris feugiat urna dolor, sit amet fringilla mi accumsan sit amet. Vivamus sit amet dictum quam, vel condimentum lorem. Cras elementum eu massa lobortis pharetra. Quisque fringilla volutpat mauris, sed feugiat velit accumsan sed. Nam venenatis nec justo eget consequat. Donec vitae lorem hendrerit nulla pulvinar eleifend. Curabitur ut odio posuere ipsum faucibus commodo sed a est. Quisque tincidunt augue nec mollis volutpat. Maecenas elementum tempus velit vitae condimentum. Integer fermentum mi sapien.' as data1,
   'Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam at elit enim. Mauris et sapien vitae nulla convallis maximus non sit amet nunc. Sed quis consequat eros, nec suscipit nulla. Suspendisse potenti. Duis a posuere dui. Donec sollicitudin eget enim sit amet hendrerit. Nullam neque nisl, sodales ac magna suscipit, iaculis interdum lectus. Integer vehicula egestas augue, finibus commodo urna feugiat et. Integer cursus, lorem ut convallis fringilla, dui elit posuere turpis, nec porta erat ligula quis tortor. Mauris imperdiet ipsum diam, sed eleifend elit blandit sit amet. Fusce eu porttitor elit. Cras tempor orci quis vestibulum tincidunt. Maecenas auctor metus eget diam pulvinar, eget commodo augue pellentesque. Integer congue vitae tellus ut sollicitudin. Nulla non augue ligula.' as data2
FROM sysobjects s1, sysobjects s2, sysobjects s3;

Add a small set of rows to be searched for
SQL
INSERT INTO FullTextTest (textcol1, textcol2)
SELECT 
   'Fusce et ligula vel dolor maximus maximus non eu urna. Aenean condimentum sem diam. Morbi cursus metus sed nulla ornare blandit. Fusce ut faucibus quam. Etiam fringilla convallis mi, hendrerit hendrerit sapien volutpat quis. Suspendisse tempor viverra congue. Sed eu velit maximus, maximus sapien id, fermentum felis. Vestibulum rutrum leo felis, ut sagittis risus efficitur sit amet. Phasellus tempor tellus vel placerat scelerisque. In hac habitasse platea dictumst. Nunc tincidunt elementum justo, vel dapibus felis pellentesque non. Nulla mattis lorem eget metus dictum, sed condimentum ipsum sodales. Nunc pretium efficitur suscipit. Proin facilisis molestie erat nec gravida. Suspendisse potenti. Ut nunc nisi, vulputate venenatis enim sit amet, consequat elementum tortor.' as data1,
   'Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam at elit enim. Mauris et sapien vitae nulla convallis maximus non sit amet nunc. Sed quis consequat eros, nec suscipit nulla. Suspendisse potenti. Duis a posuere dui. Donec sollicitudin eget enim sit amet hendrerit. Nullam neque nisl, sodales ac magna suscipit, iaculis interdum lectus. Integer vehicula egestas augue, finibus commodo urna feugiat et. Integer cursus, lorem ut convallis fringilla, dui elit posuere turpis, nec porta erat ligula quis tortor. Mauris imperdiet ipsum diam, sed eleifend elit blandit sit amet. Fusce eu porttitor elit. Cras tempor orci quis vestibulum tincidunt. Maecenas auctor metus eget diam pulvinar, eget commodo augue pellentesque. Integer congue vitae tellus ut sollicitudin. Nulla non augue ligula.' as data2
FROM sysobjects s1;
-- 119 additional rows added

Update statistics and see the overall count
SQL
UPDATE STATISTICS FullTextTest;

SELECT COUNT(*) FROM FullTextTest
-- total 3'370'437 rows

Create the full text indices
SQL
CREATE FULLTEXT CATALOG FullTextTestCatalog AS DEFAULT;  

CREATE FULLTEXT INDEX 
ON FullTextTest (textcol1, textcol2)   
KEY INDEX pk_FullTextTest;  

SELECT
   c.Name,
   CASE FULLTEXTCATALOGPROPERTY(c.name,'PopulateStatus')
	   WHEN 0 THEN 'Idle'
	   WHEN 1 THEN 'Full population in progress'
	   WHEN 2 THEN 'Paused'
	   WHEN 3 THEN 'Throttled'
	   WHEN 4 THEN 'Recovering'
	   WHEN 5 THEN 'Shutdown'
	   WHEN 6 THEN 'Incremental population in progress'
	   WHEN 7 THEN 'Building index'
	   WHEN 8 THEN 'Disk is full. Paused.'
	   WHEN 9 THEN 'Change tracking'
	END AS Status
FROM sys.fulltext_catalogs AS c

-- WAIT UNTIL POPULATION FINISHED
-- since this is initial population
-- this takes awhile, like hours

Test LIKE query
SQL
-- LIKE
BEGIN
   DECLARE @word varchar(100);
   SET @word = '%rutrum%';

   SELECT ftt.* 
   FROM FullTextTest ftt
   WHERE ftt.textcol1 LIKE @word
   OR    ftt.textcol2 LIKE @word;
END
-- Result is 119 rows
---- time 1 hour 8 minutes
--Table 'FullTextTest'. Scan count 1, logical reads 676596, physical reads 3, read-ahead reads 676591, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
-- SQL Server Execution Times:
--   CPU time = 3162688 ms,  elapsed time = 4079922 ms.

So little bit over 1 hour for the query to finish and the size of the result set is 119 rows.

Now test CONTAINS query
SQL
-- CONTAINS
BEGIN
   DECLARE @word varchar(100);
   SET @word = 'rutrum';

   SELECT ftt.* 
   FROM FullTextTest ftt
   WHERE CONTAINS(ftt.textcol1, @word)
   OR    CONTAINS(ftt.textcol2, @word);
END

-- Result is 119 rows
-- time 2 seconds
--Table 'FullTextTest'. Scan count 0, logical reads 737, physical reads 1, read-ahead reads 27, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
-- SQL Server Execution Times:
--   CPU time = 0 ms,  elapsed time = 1739 ms.

Much better, 2 seconds, the same result.

Hopefully this clarifies the situation.
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900