Everything you need to know about completing test cases
Important Note: The images contained in the document are examples, this may vary depending on the assignment!
1. Open the invitation email
2. Pay extra attention to the following elements:
- The assigned device: each assignment has to be completed on a specific device with a specific OS, these will be mentioned in the invitation received.
- You have to execute the test case on the mentioned device running on the mentioned OS
- Important: If you don’t have that device anymore or if you made an update to the OS please let the Cycle Manager know by replying to the invitation email (using the “Reply all” function).
- Please keep in mind to use only the device mentioned in the invitation.
- The assigned Deadline:
- The invitation is usually sent based on your availability stated in the registration form.
- In order to be paid, you will have to finish the test cases before the deadline. If you have any problems or questions please contact the Cycle Manager (by replying to the invitation email (use “Reply all” function) with any questions).
- If the received deadline is not in your timezone you can use the following tool to convert it: https://worldtimebuddy.com
3. Confirm your participation by replying to the email using the “Reply all” function
Once you have concluded that you are available within the given deadline and you own the assigned device you have to Confirm your participation by replying to the email using the “Reply all” function.
This step is very important – if we have your confirmation in a couple of hours you will not be replaced with another tester!
If you don’t want to / if you can’t participate (no worries), please send a message notifying us that you withdraw your participation, so we can assign another tester to the task.
4. Make sure to thoroughly read the provided Tester Specification
The Tester Spec can also be found in the received invitation.
This contains all the details about the project: info about the app, the link to the app, the bug tracker, the Known Issues list and additional details.
Please make sure to read very carefully the Tester Spec before you start to run the test cases and familiarize yourself with the testing instructions, the scope of the test and any requirements related to the application or bug reporting procedures!
5. Download the application and check the Known Issues list from the provided tester spec:
This document contains all the bugs that have been previously reported. There is no need to submit them again!
- If you encounter a bug that is present in the Known Issues List, use the BUG ID of the known issue in your test case execution.
- Reported issues that can be found in the Known Issues List will be rejected as duplicates and will not be eligible for payment.
We strongly recommend to always check the Known Issues List before submitting a new bug report!
Please note that some assignments might not have a Known Issues List. In that case, you can submit any issue you find, as long as they are within the scope of the test and not reported during the test cycle by another tester.
7. Access your assigned test case document
- In the “Steps to reproduce” Column, you will find the steps you have to follow
- In the “Expected Result” document you will find the expected results of the actions you make based on the previous column.
- Note that the Expected Results are numbered according to their respective required step.
- For Example:
- For Action number “1” you should find the Expected Result “1”
- For Action “3” you will have to check if all the sections listed under Expected Result “3” are displayed
- For Action “4” you have to make sure that Expected Result “4a”; “4b” and “4c” are all completed
8. How to complete the Test Case Document
First thing you have to keep in mind when executing the test cases is that you have to follow only the instructions listed in the test case document unless specified otherwise!
Usually, issues that are not submitted based on the instructions of the test cases are considered to be out of scope.
Test cases can have four statuses:
If you have followed all the instructions from the “Steps” column and found all the results listed in the “Expected results” it means that there are no issues with that test case and it is passed.
If you have followed the instructions from the “Steps” column and found different results than the ones listed in the “Expected results” it means that test case is failed.
Follow these steps in the case of a failed Test Case:
- Search for the issue in the known issues list and in the issues that were already reported in the tracker. If the issue is found in any of these, you will not have to create a new bug report (if you will create a new bug report your issue will be rejected as a duplicate).
- Mark the step as “Failed” and just add the Bug ID and the title of the bug in the Test Case document’s “Bug ID” column. Put the Bug ID if the code displayed in front of the title.
- If you don’t find the issue in the known issues list or in the issues that were already reported in the tracker, please submit a bug report, mark the steps as “Failed” and add the Bug ID and the title of the bug in the “Bug ID” column
If there is a special requirement that you can not complete due to reasons that are independent of you, mark the Test Case as Skipped and add the reason for skipping it into the “Comments” tab.
- Keep in mind: before marking a Test Case as skipped it’s best if you ask for the Cycle Manager’s advice on the situation (reply to the invitation email)
- An example of this situation: you run a general set of Test Cases but you know from the Tester spec that a section is not available in your Country. You can mark that test case as skipped, based on the received instructions.
- Blocked/CNT(can not test):
If you can not execute the instructions from the Expected Result column due to an issue reported for a previous TC entry that makes it impossible to access a section mark the test Case as “Blocked” based on that bug ID.
- For example, the first test case instructs you to access the settings and while executing the test case the app crashes. The next test case instructs you to reach a subsection of the Settings. The second Test Case will be Blocked based on the bug submitted for the first test case.
+1: Useful tips!
- Make sure to always confirm or decline your participation in the test case by replying (Reply All) to the invitation
- If you have any questions or if you have any kind of trouble in regards to the test case execution, reach out to the Cycle Manager by replying to the invitation email (Reply All)
- If you confirmed your participation please do not abandon the execution as this may result in you receiving fewer invites in the future.
- Please make sure to complete the test cases only on the Test Case document shared with you in the Invitation email.
- In case the test case contains more than one Tab you have to complete all of them.
- Always check the Known Issues List and the bugs reported in the current tracker before submitting a bug.
- Please keep in mind that a test case execution cannot be considered valid as long as the issues submitted based on the Test case steps have the “Waiting for tester” Status.
- After you have completed the test case and the cycle, if you receive a request based on your execution please make sure to answer that request.
- Please keep in mind that a test case execution cannot be considered valid as long as there are requests from the Cycle Manager that are still Pending.