There are a few types of feedback you can get from playtesters, and I wonder how to best gather data for each of them...
Crash Reports. When my C++ game crashes while someone is playing it, how do I best make sure that I know about it and even better... what caused it? Even getting something as simple as the file and line number which caused a crash would be incredibly helpful.
Design Feedback. When a play tester is playing the game, how can I figure out if they are having fun, why they are having fun, why they aren't having fun, and what we should spend time adjusting?
Answer
I'm assuming you're talking about on-site playtesters and not internet beta testers.
Rule #1: Don't help them. Frustration should be the top thing you should be checking for. The ideal situation would be a two way mirror with your team on one side and the playtester on another with one video camera on their face and another on the screen. Obviously this isn't feasible for most people, so do the best you can. Just having your designers sit and watch where people get stuck is very useful information. You're not going to be standing over their shoulder when they take the game home, so you giving advice on how to pass certain sections isn't going to give you the information you need. Edit: another way of putting it is this: Don't think they're "Playing it wrong"
Rule #2: Don't give them what they want. After a playtest session you have some kind of questionnaire that they fill out. The specific suggestions they have are usually not wise to take at face value. Usually there's some root cause that is triggering most responses and they just don't know how to express it. If you can figure that out, you'll be better off doing it. Although at the moment I'm having trouble coming up with specific examples.
Rule #3: Data is king. If you can (and this is really another wishlist item, honestly), track everything you can. Track where players die. Track where they run out of ammo from a specific gun. Track what pickups they miss. Track what upgrades they buy. Track what enemies do damage to them. Obviously these are FPS-specific examples, but I'm sure there are domain specific ones for whatever game you're doing. If everybody's doing something or not doing something, those are usually areas that you should spend a little more time looking at.
Basically, you don't care what player's think. You care about getting raw numbers for what players do. You need virgin eyes to see your game and tell you what makes them frustrated and what they're being led to do.
For crashes, investigate minidumps. They're not perfect, but are a very useful tool to figure out where crashes are.
Also consider a built-in bug reporting tool. Something where the user can take a screenshot , add a description, and email it to someone automatically from inside the game. Ideally with a snapshot of the world (i.e. quicksave or some kind of memory dump) if your game supports it.
No comments:
Post a Comment