|
We are looking a change on the code to make it watchable and improve it's performance.
Want to change this piece of SQL query into a loop version which
Could `Select-Insert` records by 1000 items per iteration.
On each iteration want to get a print to see what was the last inserted item
and if possible get the elapsed time on the iteration.
The Code :
INSERT INTO [tData2]
(
[Key],
Info1,
Info2
)
SELECT
[Key],
Info1,
Info2
FROM
[tData]
-- Conditions were removed as weren't related to this question
your help is really appreciated.
modified 19-Jul-12 3:50am.
|
|
|
|
|
Performance will suffer if you're going to print each item individually. Insert 50 (or so) records, see if more than a second passed, and write to the terminal that you've done the next batch.
Bastard Programmer from Hell 
if you can't read my code, try converting it here[^]
|
|
|
|
|
Limiting the loop will slow your performance. There really isn't a nice way of doing it without resorting to pre-import queries or using a cursor. Neither choice will result in faster performance.
Proper indexes on the conditions will improve performance given the query you listed.
If you are moving lots of data into an empty table and you can fully control its integrity, then remove all constraints (except identities) and indexes on the new table - update the table - and put them back on. This can greatly speed up a large insert.
You can also take a look at BULK INSERT if your SQL version will allow.
|
|
|
|
|
@Michael thanks for the info,
I recently made a solution for this used some of your previous advice,
also will consider your recent advice as it's needed.
|
|
|
|
|
Hi
I have a requirement to migrate the data from my standard OLTP database into an OLAP database based on the firing of a particular event. There will be some calculations performed on the OLTP data and the calculated/aggregated data will be inserted into the OLAP database.
I have 2 options to do this.
1. Use a web service to insert data into the OLTP database, make required calculations and then insert the data into the OLAP database from within the web service itself.
2. Use SSIS to migrate the data from the OLTP database into the OLAP database.
I am confused as to which option is better in terms of performance, security, maintainability as well as scalability of the application. My application deals with huge amounts of data which is required to be used for generation of reports.
Thanks
|
|
|
|
|
No you have only 1 option - you have to use SSIS. A web service is for publishing data to be consumed by a client, SSIS is an ETL tool, your job is ETL.
The above description is a simplification of both tools but is essentially correct. This is especially true as you are transforming large volumes.
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
Member 9270499 wrote: based on the firing of a particular event
Does this mean that the OLAP db have to be kept up to date immediately when the OLTP is updated? Or can it be updated in a batch?
|
|
|
|
|
Hi,
when i run the url http://localhost/reports i n my browser, i get the following error message:
The underlying connection was closed: An unexpected error occurred on a receive.
Home
Can anyone help me to solve that?
Note that I'm running iis 5.1 and both reports and reportserver virtual directories exist in my iis.
Cheers
|
|
|
|
|
Hi all,
I would like to delete a record in a SQL database after some time had passed if a certain field in that record has a value of No. For instance I would like to delete a record if the value of the Date field in that record is 5 days passed the timestamp date and the Returns Email field remains No.
How do I do that using the Job Scheduler in SQL? I just thought that if I can use the Job Scheduler instead of writing a stored procedure, it would save time. Thanks in advance.
|
|
|
|
|
The script below should help. Just modify what is required. Job step as you already know will be T-SQL
DECLARE @DaysPassed INT
SELECT @DaysPassed = DATEDIFF(DD,GETDATE() - 5, GETDATE())
IF @DaysPassed = 5
BEGIN
SELECT GETDATE()
END
ELSE
PRINT 'I love Code project'
GO
Cheers
|
|
|
|
|
DELETE
[TableName]
WHERE
[TimeColName] < DATEADD(dd,-5,GETDATE()) AND
[EmailColName] = 'No'
|
|
|
|
|
Want to insert all the Rows of a table in a new table with a renaming activity. Also we have some exceptions that we shouldn't move them.
Renaming Act : Change the char '_' into ' '
Exceptions : Items that if we rename they would be duplicates in the new table.
So all the items there would be unique and renamed version of our previous table.
- Here we don't have any integer ID's,
- The database is SQL Server CE
---
* Our Tables : Product, tblNew
* Each with same columns : "Product", "col1", "col2"
Want to move the records to "tblNew" , our key column that we don't want duplicates is "product"
|
|
|
|
|
so your purpose is to move rows from one table to another
but without duplicates
Your Unique key is Product Ok
Now My question is If duplicate found depending on Product column then which row should replicate
Please provide the logic...
|
|
|
|
|
seems you didn't get the question,
We need to move rows without any duplication.
But From the beginning there is no duplicates, Right ?
During our movement, we want to change a char '_' into ' ',
and this will cause some duplications , ok ?
so we want to avoid this,
How ?
when moving the rows we don't move 'product's which will be a future duplicate after this renaming :
Imagine in our products list we have :
Baby_Doll ( a future duplicate ) and Baby Doll,
we have lots of other Baby_Brush, Baby_toy1 , Baby_toy2 , ...
( These are not future duplicates )
|
|
|
|
|
Use a cursor/while loop, probably easier with a cursor.
get the field to be renamed into a variable
test if it exists in the target
insert or ignore base on the test
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
@Mycroft Holmes,
seems to be a productive way,
but could you please give the 5 lines of code you are talking about,
please also re-consider it's SQL CE, cause it lacks near everything !
Also for clarifying more of this situation please look at my previous reply on this thread,
|
|
|
|
|
SkyRunner wrote: but could you please give the 5 lines of code you are talking about,
Nope, you write the code and when you get stuck comeback and ask for help.
As it is sql ce you will probably have to do this in code, the same logic applies.
Get the original table into a list<>
get and empty list<>
loop through the data inserting the unique records into the new list<>
write the new list<> back to the new table in the database.
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
here I rote tens on thousands lines of code which are working,
about this subject I asked I'm stuck that I've asked,
I was a bit away from dealing with pure CQL scripts, and found SQL CE a real big hassle .
Here I rote lots of scripts with similar errors, I can't do a select insert query and so on !
may I'm wrong on some pieces in each code I tried,
Here I was working on this :
INSERT INTO Table1(Column1)
SELECT REPLACE(OriginalColumn, '_', '')
FROM Table2 t2
LEFT OUTER JOIN Table1 t1 ON t1.Column1 = REPLACE(OriginalColumn, '_', '')
WHERE t1.Column1 IS NULL
but still couldn't make it to meet my Tables and columns,
What about this, or the situation I've explained ?
|
|
|
|
|
Also forgot to mention, The process was really heavy, now just lloking into sql script to do the final job and do not want to do it via code.
|
|
|
|
|
First, find out if this is really a big issue:
SELECT
REPLACE([ProductName],'_','') AS NewName,
COUNT(*)
FROM
[Product]
GROUP BY
REPLACE([ProductName],'_','')
HAVING
COUNT(*) > 1
You may find that you have only a few rows that you can manually update/delete prior to the move.
Next - what are you going to do with the new duplicates? Are you not bringing them over or do you want to generate a unique name for them or are you going to do some fancy merge of the other columns/linked tables?
|
|
|
|
|
we are making/filling a new table and after filling that with the desired table, the previous table is a subject to delete, so will not do anything regarding the duplicates ( They stay to be deleted )
Do you know how can I use Select / Insert here to accomplish this task and bring all the columns data with a condition checking about preventing the insertion of future-probable-duplicates ?
There are about 180,000 rows which we will not move about 10,000 of it,
The database is generated by code, and everything was based on decision and observation.
|
|
|
|
|
Something like this should work:
INSERT INTO [NewProductTable]
(
[NewProductName],
[NewCol1],
[NewCol2]
)
SELECT
REPLACE([ProductName],'_',' '),
[Col1],
[Col2]
FROM
[Product]
WHERE
(REPLACE([ProductName],'_',' ') = ProductName) OR
(REPLACE([ProductName],'_',' ') NOT IN (SELECT [ProductName] FROM [Product]))
I want you to understand that any child records (think foreign key) based off the ignored rows will be orphaned in the new structure.
|
|
|
|
|
I think the logic should be almost right,
except a piece which I have a little doubt in the 'Where' condition
in addition to my never-ending doubt on SQL CE,
let me try it and give you the feedback of your code,
Thanks @Michael Potter, your help is really appreciated,
that is the last step regarding the database
and I hope it could help on the objective desired.
|
|
|
|
|
Hi,
Some time back my machine (Win XP) got renamed from M4B-11-720-GXP to M4B-06-425-GXP. While inatalling MS SQL Server 2005 SP3, the installer still shows me the old Reporting Service instance name - M4B-11-720-GXP\ReportServer. As a result installation fails on the authentication windows since there is no such instance present on my machine. Is there any way to get rid of this issue? From where does it load the old instance name? Can we change that? Also is there any way to unselect the "Reporting Services" option from the previous screen?
Your prompt response on this would be highly appreciated.
Thanks in advance.
Regards,
Vipul Mehta
modified 16-Jul-12 5:04am.
|
|
|
|
|
i am bigineer plese give me the login page code
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
