|
Hi All
I have been tasked with migrating an asp\db app from IIS5 to IIS6. The app basically asks users to fill in a search form which then returns relevant records. If the result returns more than about 500 records then the page fails to load (page cannot be displayed error). There are no errors in the event logs, IIS logs or the httperr log. With fewer records then it works fine. It all worked fine on the IIS 5 box.
If I remove 4 of the displayed fields (there are about 12 in total) by just deleting the field from the code below (i.e. deleting every thing between the <tr> </tr> tags) so that there are fewer fields to display for each record then it works again. Looks like it's complaining about the amount of data returned but it's only a few hundred records.
Can anyone shed any light?
Thanks for your help
Mike
The code that displays the records:
.
.
.
'create a recordset
set sqlResults = server.createobject("ADODB.recordset")
sqlResults.open selectPart, dBconnection, 1, 3
CountRecords = sqlResults.recordcount
%>
<font color="navy" face="arial">
<P></P>
<center>
<table align="center" border="2" width="589" bordercolor="navy"
bgcolor="#e6e6e6">
<TR>
<TD width="571" height="26" valign="top" align="middle"> <font
color="navy" size=+1><B><%=CountRecords%>
record(s) held</B></font> </TD>
</TR>
</table>
<%do while not sqlResults.EOF%>
<table align="center" border="2" width="589" height ="95%"
bordercolor="navy" bgcolor="lightblue" style="HEIGHT: 103px; WIDTH:
589px">
<TR>
<TD align="right" valign="top" width ="199"
bgcolor="lightblue"><font color="navy"><B>
Part No: </B></font></TD>
<TD align="middle" width= "372" bgcolor="#e6e6e6"><font color
="navy"><B><%=sqlResults(" Part No")%>
</B></font></TD>
</TR>
<TR>
<TD align="right" valign="top" bgcolor="lightblue"><font
color="navy"><B>Manufacturers
Part No:</B></font></TD>
<TD align="middle" bgcolor="#e6e6e6"><font color
="navy"><B><%=sqlResults("Manuf Part No")%>
</B></font></TD>
</TR>
.
.
About 10 more fields listed here in the same way as the above ones.
.
.
<%
sqlResults.MoveNext
loop
%>
</TABLE>
|
|
|
|
|
How can i manage database transactions between layers? Using Presentation, business and data layers. One solution i found is to declare transaction in the business layer then pass it as parameter to the DAL, but i think that's the worse i can do. Any ideas?
Never argue with an idiot. They drag you down to their level, then beat you with experience. - Dilbert
|
|
|
|
|
The DAL deals with the transactions. From the business layer you can call the DAL's StartTransaction() method (or what ever you call to choose it) and if everything goes okay you can call the DAL's CommitTransaction() and if not RollbackTransaction()
Remember to ensure that the Commit/Rollback-Transaction() method calls are in a finally block in case something goes wrong because you want to ensure that they are called.
|
|
|
|
|
Thanks Collin , someone told me that i can use the transactionscope statement.. I think this would solve my problem. By the way, i read your sql injection article and is one of my favorites. Thanks again.
Never argue with an idiot. They drag you down to their level, then beat you with experience. - Dilbert
|
|
|
|
|
Hey
I have developed a webpage that has a search feature. Now I have read articles in the past that warn about security - how hackers could type bad input that could drop or damage your tables, and ways to go about stopping this from happening.
Now does this still apply today with .net v2? As i have noticed it has some built in security that stops you from entering ceratin characters.
If I still should add extra security does anyone know of any good articles or have an pointers?
Thanks!
|
|
|
|
|
|
OK - don't use inline SQL. Use Stored Procedures.
HTMLEncode your input.
Verify the input to detect common injection conditions.
Colin has a good article on how to do this.
Arthur Dent - "That would explain it. All my life I've had this strange feeling that there's something big and sinister going on in the world."
Slartibartfast - "No. That's perfectly normal paranoia. Everybody in the universe gets that."
Deja View - the feeling that you've seen this post before.
|
|
|
|
|
Hi all,
I have a question on designing the database for my project. Here is the story. I am using VC++ connecting to SQL Express 2005. I have several tables that will have between 1000-5000 new entries a day. Will the "AddNew()", "Edit(), and "Find()" functions get slower on those tables as the entries grow beyond, let say, one million? Should I create one table per year (but then I need to run queries on each table, which I prefer not to do)? One note, these tables has indexes, of course.
Other questions that I have:
- which one is faster: FIND/FINDFIRST or SEEK?
- (VC++) Is querying "SELECT" as SQL command much faster than running Find/Seek thru _RecordsetPtr? Afik, both are done by DB engine on the server side, not by the client side, correct?
Could anyone please give me some light on these? thanks.
|
|
|
|
|
If the tables are indexed, then as your table grows, the insert (and sometimes the edit) operations tend to slow down - this is because the database needs to manage the indexes during the insert. However, a million rows (on a properly indexed table) should not be a problem for a select statement. The referse holds true on a non-indexed table - inserts tend to remain fast but select operations slow down.
There are a couple of design patterns that are applicable. One is to create multiple tables, one per month, one per year, whatever is necessary. This keeps the indivual table size small. You would then create a VIEW to represents a union of all your tables so your select statements always pull from a single entity. Another design pattern is to keep all of your data in a single, indexed, table but have a separate, non indexed, table that acts as an insert queue. New data gets inserted into the queue and that data is moved into the permenant table on a nightly basis (inerted into permenant table, delete from queue). You would still need a VIEW to provided a union between your queue and the permenant table, but you don't need to update the view as new tables are added.
|
|
|
|
|
Thanks a lot for your input. I think I prefer your second option (using a queue), although I have no idea what a VIEW is. I am pretty newbie in DB. What is the quickest/simplest way to copy data from one table (queue) to another (master)? Do I need to use stored procedure (which I don't know yet)?
Since my program will be run by multi-users in a 24/7 warehouse, I cannot do nightly transfer. So this i what I think: have an idle counter, if it hits (let's say) one hour, then that workstation start transferring one entry at a time (checking if it's still idle after each transfer). Of course I need a "setting" table in database indicating that "transferring in process" so that any other 1-hour-idle workstations won't try doing the same thing.
Will this work? Thanks again for any help.
|
|
|
|
|
This isn't a trivial process. Though simple on the surface, there is a lot you need to think about. First, transfering one entry at a time will hurt performance. You would be better off ignoring the queuing concept entirely and just inserting directly to your main table. The general logic you want to follow is this:
Start the transaction
insert from queue into master
delete from queue
commit transaction
syntax for inserting from one table to another is:
INSERT INTO TABLE2 (COL1, COL2, COL3) SELECT COL1, COL4, COL7 FROM TABLE1
Use a lock hint to lock your tables until the transaction is completed, so for example you might have the following code:
BEGIN TRANSACTION<br />
INSERT INTO TABLE2 (COL1, COL2, COL3) SELECT COL1, COL4, COL7 FROM WITH (TABLOCKX)<br />
DELETE FROM TABLE2<br />
COMMIT
For performance purposes, you may also want to drop the indexes in your main table prior to the insert and then recreate them after the insert.
|
|
|
|
|
Won't it take a long time to re-create index after deleting it?
After re-thinking about this, I started to wonder: how beneficial is it to use a temp queue? How slow is it to insert an entry to one-million-entry table? Do you have any approximate number?
If it's still acceptable (maybe less than 10 seconds), than I'll just drop this queue idea. I will still create one table per year, though.
Thanks for all your help.
|
|
|
|
|
Typically, anything greater than a few milliseconds is considered poor performance. You can easily achieve sub-10 second performance on an insert in a million entry table - assuming that your server is appropriately scaled for the job. For a million record table, you should still be able to achieve sub-second inserts. Make sure you do the math on your performance analysis. For example, if you shoot for 10 seconds in your insert time:
10 seconds * 5000 inserts/day
= 50000 seconds/day spent inserting
= 13.8 hours/day spent inserting records
Create your table, inject a few million records into the table and then do some performance analysis. If the performance is acceptable, then you have your answer!
|
|
|
|
|
I have to gather some info from and database - now the issue comes into play with either coding somethng or writing a extremely difficult query - what to do???
Here is the scenario:
Data from a single table.
Have to calculate columns(sum) according to to certain criteria and have a total at the bottom. Also have to calculate columns(sum) that subtract from a main column and then sum that column. I am just confused as to which: 1) is eaiser ( as the sytem does not have many users) 2) more efficient 3) which would you guys do (code or sql query)? 4) Is a query even possible?
Any suggestions or comments in the right direction would be greatly appreciated.
Thank You.
|
|
|
|
|
A Stored Procedure should do the trick nicely (and more efficient, as you don't need to bring all those records on the client side except the sums).
In your SP, calculate the sum of the main column, calculate the sum of the side columns, and then return the sums that you need through output parameters of the SP.
"A democracy is nothing more than mob rule, where fifty-one percent of the people may take away the rights of the other forty-nine." - Thomas Jefferson
"Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote." - Benjamin Franklin
Edbert
Sydney, Australia
|
|
|
|
|
I would create stored procs to perform the required calculations anbd return the results. SQL server is pretty efficient at calculating aggregates. In any case, this is bound to be more efficient than returning all the raw data over the network, then calculating the results. No matter how much faster your calculation is than SQL servers, it will never make up for the overhead of marshalling all that raw data over the network pipe.
|
|
|
|
|
Only 2 rows have actual data and 4 of them are just either yes (nor) no then calculate from those fileds depending and yes or no across 4 fields and only one field can be yes. Then you sum the row and that column.
Sorry to be pest - i am not very good a stored p.
|
|
|
|
|
Use Excel.
"When I get a little money, I buy books and if any is left, I buy food and clothes." --Erasmus
|
|
|
|
|
Hi,
I am trying to create a process that will take data source that has been output from a proprietary ISAM database to import into a SQL database. The thing is I want the process to be able to create the required tables based on data structure information that has been somehow encoded into the data source.
Currently the solution I am going with is to spit out a CSV file that has a header with table and data format information followed by rows of actual data that gets parsed by a SQL script however I am sure that Microsoft must have some kind of preferred solution to this kind of problem but I have not been able to find it. I have looked at the the tools that are available when creating a SQL Server DTS package as well as what seems to be available using the new Integration Services but nothing seems to be any better than the solution I just mentioned.
Anyone have any ideas, I am willing to bet there is a much better way of doing this.
|
|
|
|
|
Can the ISAM database be accessed via, say, OleDB or ODBC?
If it can, then a simple Export/Import package should do the trick nicely, shouldn't it?
"A democracy is nothing more than mob rule, where fifty-one percent of the people may take away the rights of the other forty-nine." - Thomas Jefferson
"Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote." - Benjamin Franklin
Edbert
Sydney, Australia
|
|
|
|
|
Unfortunately the ISAM database cannot be accessed in this way.
|
|
|
|
|
I am getting a "Arithmetic overflow error converting expression to data type datetime.
The statement has been terminated."
On My Sql Statement adn I can't figure out why, I have included my statement below, can anybody give me any pointers?
Declare @xmlDoc as varchar(1000)
DECLARE @DocHandle int
Declare @Loc_ORDER_REF varchar(50)
declare @loc_Order_Name varchar(50)
SET @Loc_ORDER_REF = 'Bollox'
declare @DateFormat varchar(10)
-- Get the Current Dateformat stored in l8_config
Select @DateFormat = 'dmy'--(select config_value from l8_config where config_name = 'SQLDATEFORMAT')
set dateformat @DateFormat
Set @loc_Order_Name = null
set @xmlDoc = '<Root><OPType>1</OPType><Purpose>45</ Purpose><Category>7</Category><IssueDate>23/10/2006</IssueDate><StartDate>23/10/2006</StartDate><ReviewDate>23/10/2006</ReviewDate><Brief>dzscdsxdfvszdf</Brief><IHC>1</IHC><SpecEvent>1</SpecEvent><Oedfin>6523</Oedfin><Oicfin>6523</Oicfin></Root>'
EXEC sp_xml_preparedocument @DocHandle OUTPUT, @xmlDoc
INSERT INTO OP_ORDER
(
[OP_ORDER_REF],
[OP_ORDER_NAME],
[OP_PURPOSE_ID],
[OP_CAT_ID],
[OP_ISSUE_DATE],
[OP_START_DATE],
[OP_REVIEW_DATE],
[OP_IHC_COPY],
[OP_SPEC_EVENTID],
[OP_OFF_ENT_DET_FIN],
[OP_OFF_IN_CHRG_FIN],
[OP_BRIEF_DESC],
[OP_DIV_FLAG]
)
SELECT
@Loc_ORDER_REF,@loc_Order_Name ,
Purpose,Category,
IssueDate , StartDate ,
ReviewDate ,
Case ltrim(rtrim(IHC))
when 'true' then 1
when 'false' then 0
else 0
end ,
case ltrim(rtrim( SpecEvent))
when 'true' then 1
when 'false' then 0
else 0 End , Oedfin, Oicfin, Brief, OPType
FROM OPENXML(@DocHandle, '/Root',2) WITH (Purpose int, Category int , IssueDate datetime , StartDate datetime ,
ReviewDate datetime , IHC bit, SpecEvent bit , Oedfin varchar(50), Oicfin varchar(50), Brief varchar(1000), OPType Bit )
EXEC sp_xml_removedocument @DocHandle
Select @@Identity
-- modified at 18:00 Monday 23rd October, 2006
"a fool will not learn from a wise man, but a wise man will learn from a fool"
"It is hard to fail, but it is worse never to have tried to succeed." - Theodore Roosevelt
"Success is going from failure to failure without losing your enthusiasm." - Winston Churchill
My Website || My Blogpx" targer="_blank">My Blog</a></div>
|
|
|
|
|
i have a query that returns 2 rows and i want to add 1 row manualy, so the final query would return 3 rows. can anyone help me with this?
cellardoor
|
|
|
|
|
Select your 2 rows into a view or temporary table, then add your additional row into that view/table.
Final step: select everything from that table/view.
Note: if you are using stored procedures, then temporary table will be
deleted automatically when the stored procedure is done (at least in SQL 2005)
Sincerely,
Elina
Life is great!!!
Enjoy every moment of it! 
|
|
|
|
|
select Column1,Column2 from MyTable
union
select 9000 as Column1,'whatever' as Column2
--EricDV Sig---------
Some problems are so complex that you have to be highly intelligent and well informed just to be undecided about them.
- Laurence J. Peters
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
