|
Dumb question... if you are a DBA and I came up to you and said "Here's a physical beastly box... 24 cores, ~140GB of RAM, plenty of flash storage... I need you to set up SQL 2016 on it and migrate this 10TB database from SQL 2008 R2" Assuming of course proper security and best practices, etc. Let's even "complicate" it and assume this 10TB database includes spatial data. How long would it take you? I have ZERO SQL admin experience other then slamming a few SQLs onto local machines or VMs and I think I could realistically get it setup and running in a day or two with plenty of breathing room. Yet, our supposedly experienced DBAs claim its going to take them 3 months!!! LOL... Unbelievable.
|
|
|
|
|
In the corporate enterprise in which I now find myself, three months would be quick.
Our latest "expedited" build-out took six months.
|
|
|
|
|
The last time our DBA group quoted us 3 months, it took them 9 months. I'm still "waiting" for them to complete tasks from like 3 yrs ago LOL. The team lead is horrible. So easily distracted. You basically ask him "how's it going" as a courtesy before jumping into work and you have to listen to his stories for like 30 minutes at least. Then somebody comes up to his desk and interrupts and he is distracted again for another 30 minutes. Rinse, repeat.
|
|
|
|
|
SledgeHammer01 wrote: I think I could realistically get it setup and running in a day or two with plenty of breathing room
Ee, I needed a good laugh after a day of truly dire football. Thanks for that!
Hofstadter's Law
It always takes longer than you think even when you take Hofstadter's Law into account.
I am not a number. I am a ... no, wait!
|
|
|
|
|
Depends really. Other considerations are
How it is being used
Compatible issues.
Migration window
Integrations
Connectivity issues
How the data is being migrated
Testing migrations
Etc etc
Depending on the complexities it can take the amount of time suggested.
|
|
|
|
|
Not a DBA, but the first question is whether this is for internal use or will be exposed externally. If the former, why not just do it yourself? If the latter, then it will take time, with the actual data import not taking much time at all, but security, configuration, etc. taking a while. Plus they are probably overloaded with work.
|
|
|
|
|
We are a B2B data provider, so the database is behind a firewall and the web services access it. We devs aren't allowed to touch the databases anymore. The DB team lead locked them all down. I agree on that, you shouldn't have people randomly do stuff to production databases. Due to the slowness of the database and ETL groups, most of us devs just sit around all day. Nothing to do while we wait on those groups.
What's sad is that even with all this time we give them, the databases still suck. They go down all the time, have lots of timeouts, etc.
|
|
|
|
|
Throwing hardware at the problem may speed things up, but rethinking how the data is organized is also a way to get a good boost. I still stay set it up on your own, show it works and then have it deployed into the data center.
|
|
|
|
|
Quote: the databases still suck. They go down all the time, have lots of timeouts, etc.
I suspect they are living in the DBAs' natural habitat -- an ivory tower where they can formulate their ideal environment. The result being unusable. We have that too.
|
|
|
|
|
We have 2 SQL devs that write the SPs.
One of them is fairly OK, but tbh, I think his skills are antiquated and he has no interest in learning new stuff. He only knows Microsoft SQL and tends to develop SPs using tons of temp tables and tends to "work around" issues rather then fix them.
The other dev is awful. Us devs suspect he is a former C/C++ developer who picked up a "SQL for dummies" book one weekend and decided to become a SQL developer. Just due to the way he writes SQL code. The first project they gave him, he released an SP that took 30 minutes to run!!! He actually released it like that. They've restructured everything many times over and have got it down to 5 to 10 seconds now which still doesn't meet the requirements LOL.
30 minutes to 5 to 10 seconds is a huge improvement obviously, but it just shows how bad the SQL is at my company.
|
|
|
|
|
On the bright side -- you've found your problem.
|
|
|
|
|
I feel your pain. I've recently started working on a side project of analyzing our company's database cluster for performance issues. Given this task, the company has no tools to do any kind of DB analysis , so I'm also evaluating tools to do that. During my initial analysis run, I've found that a majority of the problems are coming from bad SQL. Over half are coming from third party products the company has purchased for accounting and so forth. Fixing in-house SQL is one thing, but to get a third party company to fix their cra... um, stuff is a PITA.
modified 19-Nov-21 21:01pm.
|
|
|
|
|
It is especially trying when the third party insists that their immaculately-conceived SQL is free from original sin and it must be your loathsome environment which is responsible...
|
|
|
|
|
Yup, ran into that last week. I'm having to "build" a case showing how terrible their SQL is, like I was in a court of law. Metrics don't lie especially when they show that the SQL is running for 15 hours at a time. Also, where do they get off writing a SQL statement that starts off
Select Top 1 count(*) From ...
Good grief! 
modified 19-Nov-21 21:01pm.
|
|
|
|
|
If it was me, the SQL 2008 already was a VM, so I would shut it down, duplicate the VM and then update the SQL itself to 2016 - probably a day work, with long breaks for lunch and beer...
Skipper: We'll fix it.
Alex: Fix it? How you gonna fix this?
Skipper: Grit, spit and a whole lotta duct tape.
|
|
|
|
|
sounds like the bottle neck is not the setup but the policy environment IE corporate and convulated. And yes a day or 2 would be reasonable if it was done in isolation.
For us to get a new server up and running takes a minimum of 6 months, I have simple change requests, I mean add 3 fields to an SSIS export, that is over 12 months old and the business considers the requirement "critical".
We all work within the constraints of our environment and your, and mine, sucks!!!
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
Yup. My favorite dev story is that at one company I worked at, there was a change control board. You weren't allowed to touch code without their approval, so basically:
*user or QA reports bug
*bug goes to CCB
*CCB reviews bug and decides if its worth doing anything about it
*assuming CCB approved bug, it gets sent to dev group router
*router routes it to proper mgr
*assuming router got it right (1/2 the time they didn't) the mgr distributed it to developer for RESEARCH ONLY
*there was no ownership of code, so a bug might get routed to a person who has never touched, looked at or even heard of that component
*developer would research it, it usually took quite a while to research it, because you had to track down the original developer and ask them all sorts of questions to get up to speed
*developer would report back the finding to the CCB
*CCB would either approve, deny or request more info
*if they want more info, rinse/repeat
*only after approval could you fix it -- problem was, you general got approval MONTHS after you researched it, so you completely forgot everything about it -- no point in shelving the changes or anything, since they literally had thousands of branches, and you'd never be able to find your branch.
So anyways, due to all the red tape, it actually took me 6 months just to change a bool from true to false .
|
|
|
|
|
Thankfully I work in a departmental IT group, the core team deal with that sort of crap all the time. For me a user gets up walks over and bitches in my ear, if I own it it usually gets the job done in minutes, days at the outside. If another dev owns the app I get in his ear if the user has not gone direct to him.
We get to treat our development like a really small dev shop. It is only when we need access to the core systems and infrastructure that we hit a brick wall.
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
Joe, is that you? I didn't know the Change Control Board ever let people out of their dungeon!
Or did you dig your way out with a coffee cup and a USB key, again?
vuolsi così colà dove si puote
ciò che si vuole, e più non dimandare
--The answer to Minos and any question of "Why are we doing it this way?"
|
|
|
|
|
You can make anything last three months if you try hard enough.
Whether or not that means your efforts are pointed in the right direction is another matter entirely.
Push for an "as built" migration, where it is performed with an absolute minimum of changes, and no "improvements", being made to the data/tables/etc, along the way.
Use the words "stable", "stability", and "solid foundation" a lot, when discussing it, and keep pushing the "one thing at a time" concept -- i.e. it's less risky to move it from A to B and get it working right before implementing anything new than it is to try to do both at once.
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
I've learned not to waste my time chasing after people here. If somebody important comes asking me for status, I just say "so & so is road blocking the project" and let them worry about it. Sometimes they ask me to follow up with the road blocker to which I usually respond "I've already sent them 4 or 5 emails on it with no response, I can send a 6th email if you want..." sometimes they want me to and sometimes they don't.
|
|
|
|
|
Six hundred and fifty eight hours ... conservatively
|
|
|
|
|
To actually do a move, probably a day or so to get the new environment (machine, memory, etc.) set up and tested. Give a day to actually copy the data over. Minimally a day or two to write, prove, and run code to demonstrate to management that the data did, in fact, make it over intact (although that can then be used for other moves, so don't lose it). Going out on the proverbial limb here ... a week?
|
|
|
|
|
day 1: get sql 2016, install. oh which version did you want?
day 1-5: figure out licensing model.
day 5: ok, i think i know which licenses we need. let me raise purchase request with accounting.
day 10: oh, no we need a License advisor to come in and assess.
day 30: license advisors came yesterday. now i can raise purchase.
day 56: purchase approved
day 60: acquired licenses. now can install.
day 61: oh, we need that feature that was in enterprise edition. Ok, i will just install, oh, need different licenses.
day 62: seek payment reimbursement from License Advisor for poor decision.
day 90: all license correct. install, upgrade database. done
you getting core or user license for that machine? and if core, damn, which my company that easy with money. What over $100k.
|
|
|
|
|
I have 2 answers. Either build it yourself, or outsource it.
Now, if this was Oracle. I would say he is insane. You could restore that in oracle pretty easily and copy the config settings over pretty quickly. (restoring 10TB of data will take a while).
With that said, it always depends. Is it one schema or 200? Do they all talk to each other. When they are done, do they have to sync the stored procedures from a repository? Is it all data or data and code?
You have to cut them a little slack. 3 months seems long. Too long. Okay, way too long. I would agree with Days. I might give them a week.
But I would certainly consider hiring an outside resource to come in and do it, and also AUDIT your DB guys for best practices. I find a good Audit shakes things up a bit, and puts people on notice. It also brings in a second pair of eyes which can be very helpful, and management gets around the competing groups complaining. Hard to defend the DB guys when the outside guys say they suck!
I have done a few IT audits and it is always scary what you find. The great thing is that SOME stuff gets fixed while you are there (like default passwords on wireless routers, OMG).
BTW, it looks like you are throwing hardware at your performance problems. Usually a good sign of a design flaw.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
