|
This isn't a question about how to detect and prune unused tables in a live database. I'm more interested in the reasons why it should be done at all.
Here's some context. In our development environment we maintain a production database from which a schema and upgrade script is generated. When a database is upgraded it will create or alter existing tables on a live system, but not remove any tables.
After a debate about how it could be done and whether it should be done, I was struck by my colleague's statement of 'a lot of effort and potential pain, for not much gain'. There is of course the obvious risk of removing a table that really is used and causing a live system to fail.
There didn't seem to be much debate on whether defunct tables should be removed from a live database on the internet.
Apart from those of us who are hopelessly OCD about these sort of things, are there any good reasons for running the risk of removing defunct tables from a live system?
|
|
|
|
|
mbb01 wrote: Apart from those of us who are hopelessly OCD about these sort of things
Sorry I fall into this category so no further justification is required.
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
We use procedure that move tables into a recyclebin schema based on naming and date criteria (e.g. no live tables will contain digits in their names, but tables created/copied in response to support tickets will have the ticket number appended to the table name).
After a specified period (two weeks in our case) tables containing digits that were created prior to thespecified date are moved to the recycle bin schema. If someone screams that their table has gone, we move it back in to the original schema.
After another specified period (again two weeks), any table that has been in the recyclebin schema longer than that will be dropped.
This is all done automatically using an Agent job which calls a standard stored procedure we created.
=========================================================
I'm an optoholic - my glass is always half full of vodka.
=========================================================
|
|
|
|
|
Thanks Chris, I'll certainly put forward your good ideas to our DBAs.
But ... you answered how, not why? Implicit in your answer is the acceptance that the live database should reflect what it 'should' be from Dev.
What are the reasons (technical or business) why a live system should reflect exactly what is in Dev, when those superfluous tables are not really harming the operation of the live system?
Any insights would be appreciated.
|
|
|
|
|
Our procedures specify that is we are doing data fixes, or adding new functionality, we back up any data affected and any procedures etc to allow us to revert quickly in cases where something has gone wrong, or the client has asked for the wrong thing etc. All backups contain the ticket number - this allows other team members to find them quickly if the person who originally worked on the ticket is not available for any reason
We clear them after a set period to free space and keep the production systems as tidy as possible.
=========================================================
I'm an optoholic - my glass is always half full of vodka.
=========================================================
|
|
|
|
|
mbb01 wrote: I'm more interested in the reasons why it should be done at all.
Pruning does not prove the table is not in use or required. Imagine all your clients going down due to a simple logging-table that is only used once in the entire application, in a not-often used function.
mbb01 wrote: After a debate about how it could be done and whether it should be done, I was struck by my colleague's statement of 'a lot of effort and potential pain, for not much gain'. There is of course the obvious risk of removing a table that really is used and causing a live system to fail.
..and what is the gain? That some empty unused table is no longer present? How much space did you reclaim with that action, and how much does space cost these days? Now compare the potential gain to the potential risc.
I would strongly recommend creating some documentation; that would slow the development-proces, but it would also result in a stronger grip on your datastore. Someone wrote a SQL/Linq/EF-statement? Please update the docs and jot down which tables are impacted and when.
Bastard Programmer from Hell 
If you can't read my code, try converting it here[^]
|
|
|
|
|
What percent, by size, of the database is unused tables? If the percent is significant enough, then the savings may be using less disk space, offline backups, etc.
Is there a legal requirement on how long the data must be maintained, and, by extension, if the data is still available, can it be used in a legal sense against the company?
Are there valid reasons to prune? Yes, but they are dictated by the business needs.
|
|
|
|
|
mbb01 wrote: are there any good reasons for running the risk of removing defunct tables from a live system?
Sure. If there is a table with 100 million rows then removing it saves that space in a number of contexts. Obviously not as much an issue with 3 rows.
Additionally there is a maintenance issue. If there is an old table that Bob knows is no longer in use and then Bob dies and they hire Sharon to replace him the only way she can figure out that the table is not in use is to go through the entire code base. Not so much a problem if the is 100 lines of code but a real problem with 100 million lines of code (and poorly organized code at that.)
Same if Bob is on vacation and a new table replaced the old one but the data still exists in the old one. Sharon needs to make an emergency fix and ends up looking at the old one, and the data sort of looks correct, so that is what gets fixed.
Obviously that latter problem won't be a problem in there is sufficient testing in place to adequately test the enterprise. But if that is true then removing old cruft isn't a problem either because that same testing will demonstrate problems with that as well.
mbb01 wrote:
Apart from those of us who are hopelessly OCD about these sort of things
Me I am in the camp that there better be enough testing in place, both automated and manual, that changes have a minimal risk because changes will be needed even if old information is not removed. And I can't see that insuring functionality with additions and modifications would not cover deletions as well.
Or there is not sufficient testing in place and that is a problem.
|
|
|
|
|
Everyone seem to think this about space and performance, it should be about support. Someone looking at the schema in the future should not have to chase down the usage of redundant tables.
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
 Indeed!
Oh, and the OCD as well. 
|
|
|
|
|
What I am tring to do is to select the username, password, and role from studentstable using queryForObject.
In my JdbcTemplate syntax is
--------------Public static Object queryForObject(String sql,RowMAPPER mapper,Object ...args)----------------
<code>public class JdbcStudentDAO implements StudentDAO{
public String getLogin(StudentTO sto) {
String sql="select username,password,role from studentstable";
System.out.println(sql);
-----------------------------Below has something wrong with queryForObject-----------------------------------------------------------
Object obj=JdbcTemplate.queryForObject(sql,new
StudentRowMapper(),sto.getUsername(),sto.getPassword(),sto.getRole());
StudentTO sto1=(StudentTO)obj;
System.out.println(sto1);
return sto1.toString();
}
}</code>
-------------------------This is my RowMapper where I'm getting all rows of my database, as shown below--------------------
public class StudentRowMapper implements RowMapper{
public Object mapRow(ResultSet rs) throws SQLException {
StudentTO sto=new StudentTO();
sto.setSid(rs.getInt(1));
sto.setName(rs.getString(2));
sto.setUsername(rs.getString(3));
sto.setPassword(rs.getString(4));
sto.setEmail(rs.getString(5));
sto.setPhone(rs.getLong(6));
sto.setRole(rs.getString(7));
return sto;
}
}
-----------------------------------This is an abstract method in StudentDAO---------------------------
public interface StudentDAO {
public String getLogin(StudentTO sto);
}
-- modified 13-Jul-17 5:40am.
|
|
|
|
|
LOKENDRA YADAV wrote: select username,password,role from studentstable
Not an answer to your question, but there's a more fundamental problem with your code: You're storing passwords in plain text.
NEVER do that. And don't use reversible encryption either. Store a salted hash of the password, using a unique salt per record.
Secure Password Authentication Explained Simply[^]
Salted Password Hashing - Doing it Right[^]
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Hi friends,
I need to Import Data from Salesforce into SQL Server and export from SQL table to Salesforce using SSIS, can anybody please help me what is the better option available like any .net drivers etc. Or if we can directly import into SQL table by using any query that's also fine. As I am not much familiar any help would be very great.
I am also reading articles about these from google, so far didn't find free drivers, any help would be greatly appreciated.
Thanks in advance.
Thanks,
Abdul Aleem
"There is already enough hatred in the world lets spread love, compassion and affection."
|
|
|
|
|
What you want would require a connection string and credentials to directly access their database, I can't see that happening.
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
Hi,
I have downloaded Easysoft ODBC driver, but getting hard time to setup my DSN and connect to it, any idea which is best and easy to use ODBC driver to connect to the Salesforce.com, any help or idea would be very helpful - thanks in advance.
Thanks,
Abdul Aleem
"There is already enough hatred in the world lets spread love, compassion and affection."
|
|
|
|
|
|
If you want to transfer Data from Salesforce to SQL Server, you can try Devart SSIS, but I bigger fan of cloud solutions, `cause they are easier for me (E.g. <a href="">https:
|
|
|
|
|
Hi,
I am getting the following error when I am trying to load SSIS Package using C# code:
Microsoft.SqlServer.Dts.Runtime.Package, ErrorCode : -1073676264, Description : Error loading value "<DTS:Property xmlns:DTS="www.microsoft.com/SqlServer/Dts" DTS:Name="PackageFormatVersion">6</DTS:Property>" from node "DTS:Property".
, SubComponent: , HelpFile, HelpContext0, IdofInterfaceWithError
I got this error through the Event Listener I passed into the Load method call. This code was working before but suddenly stopped Working. The changes that happened in this mean time are, I needed to change my laptop, as it is new Laptop I needed to install all Software, some of them I installed online Evaluation versions like SQL Server 2008 R2 and 2012 because they were not available in there.
Here is the code that I used to load the Package from FileSystem:
if (!string.IsNullOrEmpty(PackagePassword))
{
_app.PackagePassword = PackagePassword;
}
//DTSEventListner eventListner = new DTSEventListner();
_Pkg = _app.LoadPackage(packageFullPhysicalPath, Listerner);
_PkgLocation = packageFullPhysicalPath;
_Status = DTSPackageStatus.Loaded;
Any help would be very helpful. Thanks in advance.
Thanks,
Abdul Aleem
"There is already enough hatred in the world lets spread love, compassion and affection."
|
|
|
|
|
PackageFormatVersion 6 suggests the package was created for SQL 2012.
Are you trying to load it with SQL 2008 R2, or using the 2008 R2 libraries?
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
But now I have installed SQL Server 2012, is it because of .Net version or SQL Server Version I have installed SQL Server 2016 first then installed 2009 R2 but now I uninstalled 2016 but installed the 2012 from online Evaluation version thinking I will apply the key from company after Package starts working. Did I do anything wrong, any help would be greatly helpful thanks my friend. It bothers more when a thing starts working and then stops.
Thanks,
Abdul Aleem
"There is already enough hatred in the world lets spread love, compassion and affection."
|
|
|
|
|
Hi Rick,
I am getting the following Warnings, is this something can be helpful for me, any advice would be very helpful my friend. Thanks my friend.
Severity Code Description Project File Line Suppression State
Warning There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference "Microsoft.SQLServer.DTSRuntimeWrap, Version=11.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=x86", "x86". This mismatch may cause runtime failures. Please consider changing the targeted processor architecture of your project through the Configuration Manager so as to align the processor architectures between your project and references, or take a dependency on references with a processor architecture that matches the targeted processor architecture of your project. EncryptDecryptSecretStrings
<br />
Warning There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference "Microsoft.SQLServer.DTSRuntimeWrap, Version=11.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=x86", "x86". This mismatch may cause runtime failures. Please consider changing the targeted processor architecture of your project through the Configuration Manager so as to align the processor architectures between your project and references, or take a dependency on references with a processor architecture that matches the targeted processor architecture of your project. FileWatcherScheduleApp
Thanks,
Abdul Aleem
"There is already enough hatred in the world lets spread love, compassion and affection."
|
|
|
|
|
That suggests it's a 32/64-bit issue. You need to change your project to target "x86" instead of "Any CPU".
How to: Configure Projects to Target Platforms[^]
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Its nice to do it but what if I deploy the code in Production and there if its different OS or different version of OS like 32 bit or 64 bit etc. Should I ignore this as it is Warning only because if it messes in the Server its going put me into more troubles.
Thanks,
Abdul Aleem
"There is already enough hatred in the world lets spread love, compassion and affection."
|
|
|
|
|
If you're referencing a 32-bit assembly, you don't have a choice; you have to target x86.
A 64-bit OS won't have any problems running a 32-bit application.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Thanks my friend
Thanks,
Abdul Aleem
"There is already enough hatred in the world lets spread love, compassion and affection."
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
