Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
Blackfire1

Blackfire1's New Persons Guide To Beta Testing.

Recommended Posts

THIS IS A GENERALIZED GUIDE. Meaning that there are aspects that fit and don't fit crowfall. Every Testing is different. Every person is different. 

With the first "wave" of testing starting soon. I figure its about time I whip this thing out. *ladies scream*
So let me tell you a little about myself. I'm an ex 3d animator and an old retired member of the now defunct Online Beta Testers Guild. I've been beta testing games and programs for a little over 16 years. It's one of my favorite hobbies. I got paid once but that killed it for me. This guide is NOT for those who've have testing experience. This guide is simply a way for me to give back to those who want to do more then play a game. Giving back to each new generation. Hopefully giving a person enough knowledge that they can take it to other games. This guide is ALWAYS being refined. (If you find something that isn't up to date on how things are done in the industry please PM me so I can review and reorganize that area.) 

Thanks again folks. I hope this helps some people and I'm putting this in the right area.

 
----------------------------------------------------------------------------------------------------------------------------------
 
Congrats, you are now a beta tester. What does that mean? You my friend are one of many who are one the fore front of the battle against horrible games. Those few lucky enough to get a glimpse into the development cycle of engineered feats such as computer programs. In this basic overview I'll give what I've learned over the years of being both a professional and hobbyist beta tester.

 

Funding Types: Why will this influence?

Back several years ago there were only a few funding avenues available for devs. Meaning from MMO's to box games how you approached a development cycle was similar. There was little time to waste on what could be. Development cycles were tight and if something didn't work well enough fast enough the tech or system was scrapped. Sometimes loaded in an expansion or content patch. That hasn't really changed too much from those days. However with the advent of crowdfunding and early access it has opened up a great deal of uncontrollable variables. There are now games that no longer fit the standard testing model. Content can now be promised for a premium. Games can be sold at full price and never be finished. Keep that in mind when testing. Since the financial mold has been broken. This will exponentially alter the testing environment. If you've paid for early access remember this. Do NOT expect a working game. Hell Don't even expect it to be fun. It'll keep your hopes from being smashed.

NDA : NON-Disclosure Agreement
- This is a legally binding contract that says you will not tell, show, record, or spread any information that you may learn while testing. There are some that are as painful as "No saying you test or we take your house" (I've personally been under one like that) to any and all leaked information will issue a perma-banning to the individual.
All are legally binding when you accept them. You can be taken to court over them. More so if your "leak" cost the company money.
If you are underage, your parents in most states are 100% liable for your actions. All paid testing positions have this. Some public tests don't. Most do. 

 
Stages of the Testing Cycle.
1. Preconception/Alpha/Early Access
2. In-house Beta/ closed Beta
3. Open Beta / Soft Launch
4. Official Launch
 
1. Preconception/Alpha/Early Access
 This is the point where a game or program has been broken down into ideas and sections of development. These are then in turn generally given to their respective teams either to execute the plan or create a means to the end. This is where all the bugs are "This is not working, oh I messed up this line of code, there we go working now. 

Once you've reached a point where there are enough systems that operate as a usable product. You start Alpha. This is where the profession $7.25 - $8 dollar an hour bug finders do everything in their power to break the game. Generally the people working on the product are still testing along side everyone else.

Early Access is the elephant in the room. People paying to be apart of the development process or to give input. Good for you. However know this. EARLY ACCESS GAMES WILL SUCK. This includes most early access crowdfunded games. If you are one of the people who gets burnt out on a program with non working systems. Avoid this step. Unless its your job, then enjoy the pay check. Complaining at this stage of the development cycle does nothing but waste energy. If there is a bad system in the game. I guarantee the Devs know.
 
2. In house Beta/Closed Beta.

This is where a majority of all bug are found. As the products and systems get more complex and polish is slowly added. Things don't like to work together. Once a program is 50%-80% done you will see "Closed beta" which is for those special invited few to give a completely untrained perspective to the product. This also includes constructive ideas on making the product better and also thinking outside the box to find the truly bizarre Bugs.
 
3. Open Beta/ Solf launch.
This is now show time. Depending on the program/product this can make or break it. Games tend to get prejudged on open beta simply because this is a general show off our product time. Yes, bugs are still going to be fixed. Also this is a great time to prepare the servers for the loads its going to meet at Launch.
 
However, what is a Soft Launch? This is something you will only find in the Free-2-PLay genre. It's there the game is nearly 100% complete. However they are still perfecting it into something people will spend money for. Its where there are no longer character wipes, patches come preplanned (sometimes), and cash shops open to give those willing the ability to advance farther then others. Its still "in beta" but everything gained from the user side is permanent. The system however is free to change.
 
4. Launch.
You will find this on most AAA or AAB titles. Most have subscription free. And if it goes BAD. It can very well destroy a game. Its one of the reasons Number 3. had become very popular.
---------------------------------------------------------
 
I'm now a tester, what do I do?
At all times try to remember your not here to "just have fun." You as a tester are not ENTITLED to anything. You are lucky to be here and the best thing you can do is make sure the game your testing is the best it can be. Unless you paid to be here via crowdfunding or early access in which case enjoy your preview please be helpful. However try to play the game both as the Devs would want it played and how it shouldn't be. How is a system designed to work? Does it? Can i break it. Also think outside the box and try to exploit any type of thing you can. Its one of the only time Exploiting in the game is OK as long as it is reported for the Devs to fix. In fact don't just think outside the box, blow ti the hell up; smash buttons,  try odd key combos, ect. If your have programming knowledge try to give yourselves invulnerability be editing file. Just remember if you find it. Someone else can and they may not be as willing to give it to the Devs to fix.
 
Tips & Tricks
1.Take notes.
Write it all down. There will be times you forget things simply because of sensory overload.
Try to keep an idea of your over all experience while playing. What felt right? What felt confusing? Was there any glitches or bugs that you see instantly?
 
2. Find a bug, try to recreate it.
If you find a problem. Retry to recreate it. Make sure you remember/write down your steps you used to cause the problem in the first place. Chances are if its more then 2-4 steps not everyone has noticed it. This can also including taking pictures of error messages and explaining in detail the results of the problem.
 
3. Report bug(s).
These can come as graphical errors, typos, ability errors, gameplay problems such as not getting XP, money or rewards, ect.
Generally you will report the bugs as you see them, giving as much information as you can muster. If you can repeat it, tell your steps. Most ways to report bugs are:
 
Email - Send them a daily or weekly email of all the bugs you find.
 
In-game bug reporter - It will generally log any and all information current at that time along with note given during gameplay. Very useful for location based bugs.
 
Forums - This is the place to make your own bug thread not to mention give feedback on general aspects of the game. Does the animations look right, if there clipping, does the AI react to orders properly, does the layout of the GUI work so that its not overly confusing. This is there most of all things reported will be posted. Not to mention it generally gives a more direct contact between the Devs and the Testers. Depending on the game there may be 1 to all players having issues with a bug. If there is already a thread on the problem. Remember to add to it. If there is no thread. Make one.
-------------------------------------------------------------------------------
Common Tester Format
 
Problem: (quick description of problem)
 
Detailed Description (This is where you flood as much information as you can)
Time: (sometimes relevant if soon after patch, could indicate server problems)
 
Location during issue: (Don't say in my house. Examples: Armory, battle field, middle of logging in, ect)
 
What you were doing when problem occurred: (Refitting stuff, Buying ingame items, Fighting Monster, crafting, ect.)
 
Steps to repeat problem: (If you tried to recreate the problem, what did you do to duplicate it.)
 
Error codes/pictures would be placed after giving as much information as possible. Dev during testing are like Detectives. They need all the clues to pinpoint the suspect.

To Video or Not to Video?

Honestly I've never seen this as a superior option. Many times NDA's will strictly prohibit media being created of that game for any reason. Some times you get games like Crowfall that don't care. Personally the time it takes to compile video, edit it, compress it, and send it to the devs is not the most time efficient. Now if there are world breaking problems that are only repeatable with a very specific manner. Video is an option. Really comes down to how much time you want to use.

Edited by Blackfire1

Blackfire1's New Players Guide to Beta Testing
*This message was delivered by a lowly minion. Please return minion back to sender in original condition. Minion is expensive to replace. Minion is not a cup holder. Please do not stick minion into a blender or microwave. Minion inc. is not responsible for damages to said kitchen appliance(s). Do not Pass go with Minion. Do not give Minion $200.00. Thank You.*

Share this post


Link to post
Share on other sites

Note that there is no NDA for any Crowfall testing phase.

Absolutely, but this is a generalized guide to beta testing in general. Not limited to Crowfall. 


Blackfire1's New Players Guide to Beta Testing
*This message was delivered by a lowly minion. Please return minion back to sender in original condition. Minion is expensive to replace. Minion is not a cup holder. Please do not stick minion into a blender or microwave. Minion inc. is not responsible for damages to said kitchen appliance(s). Do not Pass go with Minion. Do not give Minion $200.00. Thank You.*

Share this post


Link to post
Share on other sites

You should include in the repro step to try to include video of the bug occuring and show the repro steps. There is no reason why youtube couldn't be used as a bug video repository.

Edited by snissel

Obsidian-ForumSignature.jpg

Share this post


Link to post
Share on other sites

Depends honestly. If someone is willing to shell out the money for alpha access (Personally unless I'm working on content, REALLY FRIGGING BORING) more power to them and they want to be there.

 

I didn't add Video to the Repro step simply because there are some NDA's that do not allow you to record. Its dumb but it is what it is.

For Crow Fall however since there is no NDA. I don't see why not. 


Blackfire1's New Players Guide to Beta Testing
*This message was delivered by a lowly minion. Please return minion back to sender in original condition. Minion is expensive to replace. Minion is not a cup holder. Please do not stick minion into a blender or microwave. Minion inc. is not responsible for damages to said kitchen appliance(s). Do not Pass go with Minion. Do not give Minion $200.00. Thank You.*

Share this post


Link to post
Share on other sites

Awesome guide! and very helpful. I am an aspiring developer myself so seeing how the testers think is a good way to get an outside-in view from a hyper-critical perspective(down to the typo kind of thing). Thanks for writing, it will certainly help me test as well.


Might I interest you in a low-interest mortgage?

Share this post


Link to post
Share on other sites

Just pointing out, beta testers are there to have fun. However they are there to have fun by testing the game. Telling people they are not there to "have fun" just results in most of the testers not testing at all. Its better if they just do random stuff and have fun than not logging on at all. And more importantly doing random stuff usually results in better chance of encountering bugs.

Edited by Eren

Share this post


Link to post
Share on other sites

Just pointing out, beta testers are there to have fun. However they are there to have fun by testing the game. Telling people they are not there to "have fun" just results in most of the testers not testing at all. Its better if they just do random stuff and have fun than not logging on at all. And more importantly doing random stuff usually results in better chance of encountering bugs.

 

So true, you'll find more bugs by just doing what you would do on release than having a check-list. 

Share this post


Link to post
Share on other sites

Hi there Blackfire! Thanks for the hard work. I'm going to editorialize some of this for ya  B)
 

My comments will have **EDIT** (comments) all in bold
 

Morning Folks.
With the first "wave" of testing starting soon-ish (**EDIT**End of summer is late August). I figure its about time I whip this thing out. *ladies scream*

So let me tell you a little about myself. I'm an ex 3d animator and an old retired member of the now defunct Online Beta Testers Guild. I've been beta testing games and programs for a little over 16 years. It's one of my favorite hobbies. I got paid once but that killed it for me. This guide is not for those who've have testing experience. This guide is simply a way for me to give back to each new generation. It is ALWAYS being refined. (If you find something that isn't up to date on how things are done in the industry please PM me so I can review and reorganize that area.) 

Thanks again folks. I hope this helps some people and I'm putting this in the right area.
 
----------------------------------------------------------------------------------------------------------------------------------
 
Congrats, you are now a beta (**EDIT**he means Alpha) tester. What does that mean? You my friend are one of many who are one the fore front of the battle against horrible games. Those few lucky enough to get a glimpse into the development cycle of engineered feats such as computer programs. In this basic overview I'll give what I've learned over the years of being both a professional and hobbyist beta tester.

 
NDA : NON-Disclosure Agreement (**EDIT**Like others have said Ace has made it very clear there is NO NDA) - This is a legally binding contract that says you will not tell, show, record, or spread any information that you may learn while testing. There are some that are as painful as "No saying you test or we take your house" (I've personally been under one like that) to any and all leaked information will issue a perma-banning to the individual.
All are legally binding when you accept them. You can be taken to court over them. More so if your "leak" cost the company money.
If you are underage, your parents in most states are 100% liable for your actions.
 
Stages of the Testing Cycle. (**EDIT** This is a key one everyone should be very mindful of. Blackfire does a solid job explaining that games in Alpha & Beta are NOT games yet.

1. Preconception/Alpha
2. In-house Beta/ closed Beta
3. Open Beta / Soft Launch
4. Official Launch
 
1. Preconception/Alpha
 This is the point where a game or program has been broken down into ideas and sections of development. These are then in turn generally given to their respective teams either to execute the plan or create a means to the end. This is where all the bugs are "This is not working, oh I messed up this line of code, there we go working now."
 
Once you've reached a point where there are enough systems that operate as a usable product. You start Alpha. This is where the profession $7.25 - $8 dollar an hour bug finders do everything in their power to break the game. Generally the people working on the product are still testing along side everyone else.
 
2. In house Beta/Closed Beta.
This is where a majority of all bug are found. As the products and systems get more complex and polish is slowly added. Things don't like to work together. Once a program is 50%-80% done you will see "Closed beta" which is for those special invited few to give a completely untrained perspective to the product. This also includes constructive ideas on making the product better and also thinking outside the box to find the truly bizarre Bugs.
 
3. Open Beta/ Solf launch.
This is now show time. Depending on the program/product this can make or break it. Games tend to get prejudged on open beta simply because this is a general show off our product time. Yes, bugs are still going to be fixed. Also this is a great time to prepare the servers for the loads its going to meet at Launch.
 
However, what is a Soft Launch? This is something you will only find in the Free-2-PLay genre. It's there the game is nearly 100% complete. However they are still perfecting it into something people will spend money for. Its where there are no longer character wipes, patches come preplanned (sometimes), and cash shops open to give those willing the ability to advance farther then others. Its still "in beta" but everything gained from the user side is permanent. The system however is free to change.
 
4. Launch.
You will find this on most AAA or AAB titles. Most have subscription free. And if it goes BAD. It can very well destroy a game. Its one of the reasons Number 3 had become very popular. (**EDIT**Or if you are EA/Ubisoft you'll ship that broken crap and charge $60)

---------------------------------------------------------
 
I'm now a tester, what do I do?
At all times try to remember your not here to "just have fun." (**EDIT** Meaning have a ton of fun but also help make CF great by focusing on the areas the devs need us too!) You as a tester are not ENTITLED to anything. You are lucky to be here and the best thing you can do is make sure the game your testing is the best it can be. (**EDIT**Some clarification here. We are entitled to whatever we paid for; to test broken systems before others lol) However try to play the game both as the devs would want it played and how it shouldn't be. How is a system designed to work? Does it? Can i break it. Also think outside the box and try to exploit any type of thing you can. Its one of the only time Exploiting in the game is OK as long as it is reported for the Devs to fix. In fact don't just think outside the box, blow ti the hell up; smash buttons,  try odd key combos, ect. If your have programming knowledge try to give yourselves invulnerability be editing file. Just remember if you find it. Someone else can and they may not be as willing to give it to the devs to fix.

 
Tips & Tricks
1.Take notes.
Write it all down. (**EDIT**If you have a second monitor I'd recommend opening up a word doc and putting notes there. Much easier to copy/paste from there.) There will be times you forget things simply because of sensory overload.

Try to keep an idea of your over all experience while playing. What felt right? What felt confusing? Was there any glitches or bugs that you see instantly?
 
2. Find a bug, try to recreate it.
If you find a problem. Retry to recreate it. Make sure you remember/write down your steps you used to cause the problem in the first place. Chances are if its more then 2-4 steps not everyone has noticed it. This can also including taking pictures of error messages and explaining in detail the results of the problem.
 
3. Report bug(s).
These can come as graphical errors, typos, ability errors, gameplay problems such as not getting XP, money or rewards, ect.
Generally you will report the bugs as you see them, giving as much information as you can muster. If you can repeat it, tell your steps. Most ways to report bugs are:
 
Email - Send them a daily or weekly email of all the bugs you find. (**EDIT**This will come down to how they want bugs submitted aka forums only or adding in-game reporting. Each dev teams sets their own systems as mentioned below. No need to spam Ace with emails if they don't want the reports that way lol) 
 
In-game bug reporter - It will generally log any and all information current at that time along with note given during gameplay. Very useful for location based bugs.
 
Forums - This is the place to make your own bug thread not to mention give feedback on general aspects of the game. Does the animations look right, if there clipping, does the AI react to orders properly, does the layout of the GUI work so that its not overly confusing. This is there most of all things reported will be posted. Not to mention it generally gives a more direct contact between the Devs and the Testers. Depending on the game there may be 1 to all players having issues with a bug. If there is already a thread on the problem. Remember to add to it. If there is no thread. Make one.
-------------------------------------------------------------------------------
Common Tester Format(**EDIT**Just wanted to give you props for this. If Ace forgets to give us a bug report format this is solid!)

 

Problem: (quick description of problem)
 
Detailed Description: (This is where you flood as much information as you can)

 

Time: (sometimes relevant if soon after patch, could indicate server problems)
 
Location during issue: (Don't say in my house. Examples: Armory, battle field, middle of logging in, ect)
 
What you were doing when problem occurred: (Refitting stuff, Buying ingame items, Fighting Monster, crafting, ect.)
 
Steps to repeat problem: (If you tried to recreate the problem, what did you do to duplicate it.)

 

Error codes/pictures would be placed after giving as much information as possible. Dev during testing are like Detectives. They need all the clues to pinpoint the suspect.

 

 


Really nice to see the community helping each other out and doing all we can to make the game better!

Share this post


Link to post
Share on other sites

Really? So I'm a premium user because I got into Alpha 2? I feel like a King now!  :P

 

Ofc it is. For years now, games (specially MMOs) are "pre-sold" via this "beta/alpha/early access" thing. It even became a category on Steam.

 

While for KS it's a little different (it's more like a "gift" to thanks people who pledged a certain amount & it fits the whole "help us make this game a reality"), for others it's more like an early moneygrabber (why bother waiting 4 years of devs, let's get the money now).

 

Funny though (nothing personal mate, this is just an observation) how every times you can see a post like that on games' forum which are in EA/alpha. Usually, people do not care; they payed to have an early access and to be able to play before others. They won't report any bugs (except huge ones obviously), even if they say otherwise. Myself included (I used to be payed to test games, so I can't really blame them when you are the one who have to pay).

Edited by krema

Share this post


Link to post
Share on other sites

Really? So I'm a premium user because I got into Alpha 2? I feel like a King now!  :P

 

It's the way things are now. Look at the recent games that offer alpha access if you just buy that 150$ founder's package. 

 

I expect CF to be a real Alpha. We'll have limited access to the server with limited access to the Archetypes.  Dev's already mentioned the limitation of gear already. I guess that we'll choose an Archetype and just try to log in. Once in, we'll try to kill each other.


etDenA9.png
Camaraderie ~ Loyalty ~ Honor ~ Maturity ~ Integrity ~ Duty

Share this post


Link to post
Share on other sites

For those who haven't tested before I really recommend finding a group of people to test it, because testing is not always easy, or fun.

 

Having people work with you to recreate bugs, and to simply test things out makes the testing experience much better in my experience.

 

It is really easy to burnout at testing, or to get frustrated because the dev's are taking things in their direction, and not in the direction you like. 

 

Having more people to test with and talk with helps you see things from different perspectives.

 

If you go in thinking I am only going to play thief, I only like thiefy stealthy classes, and I only like XYZ pvp, so I am only going to test thief, and XYZ pvp, 

 

It is really easy to burnout, or to get over attached to your ideal of what should be, and once the attitude of I'm right, and the dev's are wrong sinks in hard

 

it can make testing really difficult. 

 

So for newer testers making this a social activity, and not getting letting yourself get too absorbed or emotionally invested in a particular aspect of them game helps a lot.

Share this post


Link to post
Share on other sites

Lol

 

"Testers"

 

With the amount of complaints going on in early access games, I very much doubt that there are many testers.


 

This game looks like a larger scale version of marvel heroes so far with forts.  - nephiral marts 7 2015

 

Share this post


Link to post
Share on other sites

I'm not sure why there's always one person who comes out of the woodwork, bearing this knowledge that they think others don't know or that the developers won't tell them.


zjsbjmn.png

Share this post


Link to post
Share on other sites

I'm not sure why there's always one person who comes out of the woodwork, bearing this knowledge that they think others don't know or that the developers won't tell them.

 

Because there are some that don't know.  There are some that haven't been part of an alpha or beta test.  There are some whose online gaming experience started with WoW or after.

 

Not everyone has your experience.


http://www.twitch.tv/Waikikamukau Yes, it's Why-Kick-Uh-Moo-Cow / Twitter: @TheMukau

Share this post


Link to post
Share on other sites

I'm not sure why there's always one person who comes out of the woodwork, bearing this knowledge that they think others don't know or that the developers won't tell them.

Agreed. I appreciate people trying to be helpful but posting a bunch of common sense stuff and then being praised for it makes me laugh. I happens on every game I've tested.

 

 

Ps. Don't forget to turn your computer on.

Edited by Ziz

MOkvLlm.png

Share this post


Link to post
Share on other sites

Lol

 

"Testers"

 

With the amount of complaints going on in early access games, I very much doubt that there are many testers.

Well this isn't really an "early access" game, so don't see how that applies. I think its clear ACE will add testers as they need, whether it be from quantity or quality they will likely keep adding until they get what they need.

Edited by pang

Share this post


Link to post
Share on other sites

Ok folks many of you have seemed to missed the meaning of this post. It's ok because your perspectives are not original and these are issues that always come up when I post this no matter the game. So thank you for this opportunity to clarify. :)
 

1. This is a generalized testing guide. NOT a testing guide for Crowfall. NO two Beta/alpha tests are the same. Perfect example is there is no NDA on this game. 

That's the point of this being generalized. Something won't fit and some things will fit perfectly. Remember this is also for those who have NEVER tested before or only used "testing" as a way to pre-play the game and nothing else.
 

 

 

I'm not sure why there's always one person who comes out of the woodwork, bearing this knowledge that they think others don't know or that the developers won't tell them.

2. This is a guide. Not everyone has your omniscience knowledge base of testing. You have to learn how to crawl before you walk. Not everyone understands this idea. Thank you for allowing me to use you an en example lethality

 

3. Beta testers are all different. I never said you're not here to have fun. I said you're not here to JUST have fun. There more to the testing process then just pre-viewing a game.

Edited by Blackfire1

Blackfire1's New Players Guide to Beta Testing
*This message was delivered by a lowly minion. Please return minion back to sender in original condition. Minion is expensive to replace. Minion is not a cup holder. Please do not stick minion into a blender or microwave. Minion inc. is not responsible for damages to said kitchen appliance(s). Do not Pass go with Minion. Do not give Minion $200.00. Thank You.*

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...