If you remember, early US space flights had chimpanzees pushing the buttons in the spacecraft.
This was done to ensure that in environments of varying gravity, an astronaut could actually push the necessary buttons to control the spacecraft. Actually, astronauts demanded that they be able to "fly" the spacecraft as they didn't want to be seen as being nothing more than "space monkeys" going along for the ride and doing nothing while being strapped to their seats.
Chimpanzees were cheap substitutes for astronauts. It also proved that astronauts were really space monkeys.
PS. Everybody talks about the bravery of the astronauts but nobody says anything about the bravery of chimpanzees, dogs, fowl, etc., that went ahead of man into space!
(I don't know who said it, but is a good afirmation)
So if you can get enemy's agents, you will have the need intel to make better deploys
If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpfull answers is nice, but saying thanks can be even nicer.
On most projects that I've worked on we used this technique.
All tests were made withing the team to ensure quality and at the end, the release is made available to a small subset of end-users.
These users are usually not random. They often reflect specific behaviors and usages that we really want to make sure we have no surprises.
At the end, this whole selected set of testers represent the majority of live usage cases we might have.
And very important, they are willing testers, interested people on the functionalities, not just some randomly chosen people that won't actually test anything. Documentation is supplied so that the users know what to test and what's the expected behavior (no bible).
Software goes pretty much bug free when they reach these users (ok ok, a couple of bugs may be found but nothing critical or blocking)
Until now, from these tests, the most important information is usability feedback.
New features or UX refactorings require new ways of user interaction and we might fail or forget some small details that won't be forgotten by these testers.
Oh how I wish I could do that. Most of our users are broken. Developing applications for them is like trying to design binoculars that would fit a person stepping out of a Picasso painting. The only way to do it "right" is to mis-align it from what would work in the rest of the world.
in isolation and watch with hidden cameras. No user has discovered the software yet, but one has demonstrated the use of tools lately. He tried to crack open a nut by hitting it with an iPad. Sooner or later he will realize that it's easier to crack open the iPad by beating it with a nut
I write software strictly for in-house use in a warehouse/production environment. I'm the only developer. Typically I run a dev version by my boss but it almost always goes through testing on the production line before making the call to release it. We'll capture all the data, analyze all possible scenarios, etc. until we're comfortable with how well things are running. Doesn't usually take too long, aside from the fact that our users, like most, tend to be stupid to the fact that we can't help them if they can't recount the steps they took when they encountered a problem with the software. LoL.
djj55: Nice but may have a permission problem
Pete O'Hanlon: He has my permission to run it.
The User test is sort of the all or nothing. We are, unfortunately, a very informal shop. Our specs and use cases are just static pictures of a screen shot. Drives me crazy. Out QA is one part time person shared between 4 or so projects, even in the middle of trying to release.
We make OEM versions of our commercial ink-jet systems. We can't test a lot of these in-house, since they require a customer's printing press. We do have a couple test bench presses (one of which I call the g**d*** a*******), but they don't really look or act like the end product. As a result, most of our OEM development is based on specifications, regular communication with the customer, and whatever we can simulate locally.
that's why we have service packs and .x releases. when your users get a hold of the software they are going to use it in ways you never expected (hoped). sometimes this is a good thing and other times you will think to yourself...
"what the hell??"
as if the facebook, twitter and message boards weren't enough - blogged
Interesting choice of words. As in, we beat our software against the user's head. Probably not that far from the truth, although it can be mutual, they then beat the bug reports against the developer's head.
But it also smacks (pun intended) of a sort of "us and them" attitude. I much prefer the soft glove, the new age, touchy-feely "we test WITH the user", following the mantra of "it's so much more productive to work together as a team", barf, barf.
That said, I do experience that when testing is done with the user, and as early as possible (frankly, I see no reason not to get rid of the QA department, replacing with automated integration / acceptance tests the ornery people who don't know how to even write down accurately what they did) the result is a much more productive-enhancing product.
Then again, that requires that the user is engaged and interested and devotes time to testing. It requires that the user isn't a burnt out shell of an underpaid and overabused participant in a servitude-based social order, but is actually motivated to improve the quality of their lives and the quality of the company's process and product.
In our department we program all the machine's logics (GUI, movement control, CNC, PLC...) then, once we have the machine mounted in the workshop we start downloading our code to the controllers installed in the machine and then we start testing our job. After some time, we hand a manual copy to the guys in our company which will program the machine to execute one specific part. This is what we call the second testing stage. After that has been finished we call our customer who comes to our factory with some operators that will use the machine. We give a training to those operators and they test again the machine. After that we install the machine at the customer's facility when we do a second training to the operators and again everything is tested.
Of course then we start with the technical assistance remotely, but this is not called testing, this is normally well... another thing...
And I want a sign off in blood, I want test cases that are verified and the business owners first born as hostage. And that only after the business has verified and validated every number that come out of the dammed application.
You work for a bank, you get it right or you don't get it at all(this has been known to happen all too often).
Never underestimate the power of human stupidity
We test our own code. After that some people at the customer test our code. After that the users get to test it... In a live production environment that is
In rare occassions stuff really breaks, as in the user can not continue what he is doing. No big deal mostly. The customer calls and we have a new version in five minutes.
In really very few occassions the user tests in a test environment. And when they do they have no idea what they're doing. That's particulary funny since we build our software exactly how the customer wants it