I've seen behind the scenes of a certain, well-known, very popular MMO based here in the UK. The tools they have to use are just awful; easily the worst scripting language I've ever used, and I've used one which only supported single-bit variables. The company makes a big noise about its utterly irrelevant employee perks and awards for same -- baskets of fruit brought to your desk; games room; etc. -- to attract new people, but the place was like The Stepford Wives. (I was there for a two-day interview. Before the end of the first day, I already knew I didn't want the job; I'd wasted half a day trying to write a trivial game in their scripting language which I could have rattled off in 680x0 assembly language in about 30 minutes! I stuck it out because I was intrigued by the actual game design issues they faced and wanted to know more about it. They have some great people working there, but they're seriously crippled by the tool-chain, IMNSHO.)Nice story.
Now, this company rakes in six million dollars a *month*, but they didn't want to replace the tool-chain with something that's actually fit for purpose, and pay some interns to convert the old scripts to the new system. The developers were well aware that their tools sucked, but the powers that be weren't willing to take a little short-term pain to radically improve their employees' productivity over the long-term. Bah!
Oluseyi wrote: [Link]
It takes a tremendous amount of work to obtain operable data from user feedback, even when you're sitting there looking at the user and videotaping him/her. Many companies simply don't have the infrastructure in place for this, and for products that do not generate the sorts of revenues that Microsoft had seen for Halo and Halo 2, even a scaled-down version of the test facility may be cost prohibitive.A post with a reference to a publication detailing previous industry efforts stands out amongst the rest of the opinions.
Anyway, there's a lot of interesting posts, including discussion about the silent majority, the vocal minority and who should have what say and whether their perspective is relevant with a high signal to noise ratio.