|
Sander Rossel wrote: but this took an entire afternoon while it should've taken about an hour.
...and so if the production machine goes down or gets hacked who is the person that is on the hook for that?
Myself it has been a long time since I would even accept credentials on a production machine when there were actual ops personnel around. And I discourage handing out such access to others. If there is a problem then figuring out the source is easier when there are far few fingers in the mix.
|
|
|
|
|
But actually getting anything done becomes pretty much impossible.
|
|
|
|
|
Sander Rossel wrote: But actually getting anything done becomes pretty much impossible.
That would then be a management problem.
But I haven't experienced that situation myself. Operations that fails to meet requests do not last long. Just as any other position that doesn't do their job doesn't last.
|
|
|
|
|
Sounds like you need some sort of UAT environment between DEV and PROD.
On our team, when you make a change to UAT, you document it with enough detail that it can be applied to PROD, but someone else has to make the PROD changes based on your documentation alone.
You fix the documentation until the PROD deployment succeeds.
What happens if you need to stand up the same app for a different site, department, client, or Disaster Site?
If you want to test an application upgrade path starting with an identically built machine...
If someone wants to upgrade the OS under your app?
3 years from now it will be a lot harder to remember all of the little details that got it working. It is also likely that various things were tried on the DEV machine that did not quite work correctly... Throw DEV away after a bit and clone the UAT environment for the next round of DEV.
|
|
|
|
|
That COULD work if all software I deployed to PROD worked.
More specifically, getting the software to run on PROD wasn't the issue, getting it to run PROPERLY was.
We do have a DTAP street, but production is always a little different.
For example, it uses SSL (I don't know why we don't use that for everything, even internal services), it should sent out actual emails to actual people, it uses different certificates (because we can't connect to production services from a non-production environment), the firewall is configured differently, etc.
Sometimes you just need to do some testing on production, especially when third party services are involved (which is the case here).
But when you need someone to open a text file for you that process becomes very difficult.
I do agree that we SHOULD document the deployment process for every bit of software though (but we don't).
|
|
|
|
|
I've been on both sides of this issue. The problem is too many companies have been burned by developers releasing code that hasn't been properly tested and confirmed to work in the target environment. Just because it works on my [development] system and server doesn't mean it will work on the CEO's.
|
|
|
|
|
Well, I guess not giving developers permissions to their target platform solves that issue as no software gets released anymore!
But seriously, if untested software makes it to production then your issue is with testing, not releasing.
If anything, the chances that software doesn't work as expected is only bigger if people who know nothing about the software do the releasing.
|
|
|
|
|
Not speaking of you directly, but generally, developers in many companies do not have have the security or legal compliance mindset. This has been the case at most every company I have worked at, and unfortunately is the case at my current company.
A few months back, the new Senior Manager that is over both my team and the applications (development) team came up to me and informed me that he was making me the "gatekeeper" over all security aspects of our internal AS/400 system. I am not an AS/400 person. I have very limited experience with it as a user. I basically know how to unlock an account, reset a password, and restart a stuck print queue. Any of the AS/400 administration has been handled by the applications team for decades.
So, why am I now the security "gatekeeper"? He explained that the developers would just create a new service account with full sysadmin access for every application and every robot and every agent. Whenever anyone with a Supervisor or above title asked for access to something, they granted that access without questions asked. My Sr. Manager had discovered that one of our warehouse workers had access to just about every system, including Accounts Receivable and Payable (because his supervisor wanted him to be able to check to see if some customers were current on their invoices before shipping out).
I had to audit user access across the board. I took away access that was not properly approved and documented. Because people could not do their jobs the (wrong) way they were used to doing it, production ground to a screeching halt for almost a full week while access was straitened out, and people were re-trained to do their jobs the correct way. It cost the company thousands upon thousands of dollars. Granted, if something serious had happened and we had been found to be out of legal compliance, it could have cost millions.
I still do not have the know-how to administer the AS/400 (although I am slowly learning). So, how do we make this work? The developers must submit security requests to me for review. I properly log the requests, ensure that all the required parties have reviewed the request and approved it, then I create a work order and send it back to the developer/admins to actually execute the security change. If they are found to have made security changes without following the procedure, they are subject to disciplinary action. (It only had to happen once.)
- - - - - -
I'll go out on a limb and say that most (I did not say all) developers are focused on "Making it work" and are not focused on security and legal compliance. That is why there needs to be a separation of development from administration.
It is indeed sad that the separation is implemented so poorly at your company. It is regrettable that an appropriate separation is not yet achieved here at my company. The unfortunate reality is that giving production administration to development teams is not the way to go.
In your case, you need a procedure in place to have a copy of your production system and config pulled to a dev/test system, allow you to make the changes needed to deploy/repair whatever is needed, then pass the specific changes (properly documented) back to the admins to review and implement as a whole, not in piecemeal. That would not only make implementing the changes easier, but would also make the sysadmin's manager's job easier in determining what training or additional personnel are needed for them to give you better response time and better collaboration.
Money makes the world go round ... but documentation moves the money.
|
|
|
|
|
It's not so much about doing everything yourself, it's just that it would be really nice and no-risk if I just had access to a log file.
I don't know much about security, IIS, reverse proxies, and user management.
But I do know how to open a log file and run a query on a database.
I consider the latter to be a part of my job, and those are the things I cannot currently do.
The former can be handled by someone else.
|
|
|
|
|
I agree with you. If they had it set up properly, you would have at lease read access to the logs, unless there is something else in those logs that would be considered confidential.
In that case, they should be parsing the logs, and giving you access to the appropriate portions.
Money makes the world go round ... but documentation moves the money.
|
|
|
|
|
Thank you for your post willichan, it was enjoyable
|
|
|
|
|
Of course, if your application had been deployed in UAT or some other environment with representative security - then you could have worked through the issues on systems you had access to. Instead of futzing around in production. Then your install/deployment would have catered for all of the issues that arose.
I would also ask about whether all of the custom things you did made it back into the systems documentation - so that in five years time when someone else needs to re-install that they know to repeat the things you did to make it work.
Not suggesting that this is representative of your situation - just that there are two sides to the coin. Separation of duties is painful, but sometimes the pain is in forcing the documentation and awareness into the process. Does not sound like they way it worked for you though.
|
|
|
|
|
In the last days I'm facing a lot of codeproject.com- server down. Just at the moment, also several downs. Same for others also?
|
|
|
|
|
just got it twice... you are not alone
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
Confirmed...thought it was just me! 
"Go forth into the source" - Neal Morse
|
|
|
|
|
Yep - it's been a messy week. We had a one-two punch with hardware failures and database issues both at the same time. Hardware is sorted out, but database has taken longer and I think it will be sorted after this final update that's currently in progress.
cheers
Chris Maunder
|
|
|
|
|
Thanks for Feedback. Have a ![Beer | [beer]](https://www.codeproject.com/script/Forums/Images/beer.gif) _after_ all is fixed 
|
|
|
|
|
After?
Ah. That may explain a few things...
cheers
Chris Maunder
|
|
|
|
|
|
Roger Wright wrote: Opel
Yeah - but did it actually run right before that? 
cheers
Chris Maunder
|
|
|
|
|
Chris Maunder wrote: id it actually run right before that?
It ran great for years, way too much horsepower for its weight. I climbed mountain fire roads late at night that most 4x4 trucks couldn't pass; unfortunately, I also hammered the entire exhaust system flat doing so. The backfiring led to a clogged carb, and its ultimate demise. Sadly, I traded it in for a Fiat X-19, the single worst vehicle I've ever owned. 
Will Rogers never met me.
|
|
|
|
|
Roger Wright wrote: I traded it in for a Fiat X-19
Either you'd been drinking or hadn't drunk enough.
(though actually they seem like a hugely fun car as long as you don't actually crash into something)
cheers
Chris Maunder
|
|
|
|
|
On a level road, on a clear, sunny day, the car was a delight. Think Pacific Coast Highway in the Peoples' Republic of California, Ventura to the Oregon border with clear skies and no traffic. Yeah, that was a long time ago. The beast looked so fast, it was always a risk - cops wanted to issue a speeding ticket for just parking it. But on a slope, 45 mph, max. Typical Fiat - Design by Fisher, execution by Fisher-Price. My first clue should have been the sound it made when I opened the driver door - identical to operating a flip top on a cold beer...
Will Rogers never met me.
|
|
|
|
|
Roger Wright wrote: Fiat X-19, the single worst vehicle I've ever owned.
I knew a guy who bought one new here in Ohio. The road salt from our first winter ate the car.
Software Zen: delete this;
|
|
|
|
|
Yes _after_ please  I will Charge the bill for the beer. If I counted correclty it will be 11 beers for the whole Team. Ok let have everyone two, therefore I'm ready for a bill for 22 beers; But please _not_ in the most expensive bar in toronto.
And no, Kent Sharkey does _not_ Count!
You know my mail, I'm awaiting the beer bill and some selfies here in the Lounge 
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
