Functional Testing
Functional testing is a type of black box testing that bases its test cases on the specifications of the software component under test. Functions are tested by feeding them input and examining the output, and internal program structure is rarely considered (Not like in white-box testing).
Functional testing differs from system testing in that functional testing verifies a program by checking it against ... design document(s) or specification’s, while system testing validates a program by checking it against the published user or system requirements Functional testing typically involves five steps
1. The identification of functions that the software is expected to perform
2. The creation of input data based on the function's specifications
3. The determination of output based on the function's specifications
4. The execution of the test case
5. The comparison of actual and expected outputs
Black Box Testing:
also known as Behavioral Testing, is a software testing method in which the internal structure/design/implementation of the item being tested is not known to the tester. These tests can be functional or non-functional, though usually functional.
This method is named so because the software program, in the eyes of the tester, is like a black box; inside which one cannot see.
Black Box Testing attempts to find errors in the following categories:
• Incorrect or missing functions
• Interface errors
• Errors in data structures or external database access
• Behavior or performance errors
• Initialization and termination errors
White Box Testing:
The purpose of any security testing method is to ensure the robustness of a system in the face of malicious attacks or regular software failures. White box testing is performed based on the knowledge of how the system is implemented. White box testing includes analyzing data flow, control flow, information flow, coding practices, and exception and error handling within the system, to test the intended and unintended software behavior. White box testing can be performed to validate whether code implementation follows intended design, to validate implemented security functionality, and to uncover exploitable vulnerabilities. White box testing requires access to the source code. Though white box testing can be performed any time in the life cycle after the code is developed, it is a good practice to perform white box testing during the unit testing phase.
White box testing requires knowing what makes software secure or insecure, how to think like an attacker, and how to use different testing tools and techniques. The first step in white box testing is to comprehend and analyze available design documentation, source code, and other relevant development artifacts, so knowing what makes software secure is a fundamental requirement. Second, to create tests that exploit software, a tester must think like an attacker. Third, to perform testing effectively, testers need to know the different tools and techniques available for white box testing. The three requirements do not work in isolation, but together.
UNIT TESTING
A series of stand-alone tests are conducted during Unit Testing. Each test examines an individual component that is new or has been modified. A unit test is also called a module test because it tests the individual units of code that comprise the application. Each test validates a single module that, based on the technical design documents, was built to perform a certain task with the expectation that it will behave in a specific way or produce specific results. Unit tests focus on functionality and reliability, and the entry and exit criteria can be the same for each module or specific to a particular module. Unit testing is done in a test environment prior to system integration. If a defect is discovered during a unit test, the severity of the defect will dictate whether or not it will be fixed before the module is approved.
INTEGRATION TESTING
Integration testing examines all the components and modules that are new, changed, affected by a change, or needed to form a complete system. Where system testing tries to minimize outside factors, integration testing requires involvement of other systems and interfaces with other applications, including those owned by an outside vendor, external partners, or the customer. For example, integration testing for a new web interface that collects user input for addition to a database must include the database’s ETL application even if the database is hosted by a vendor—the complete system must be tested end-to-end. In this example, integration testing doesn’t stop with the database load; test reads must verify that it was correctly loaded. Integration testing also differs from system testing in that when a defect is discovered, not all previously executed tests have to be rerun after the repair is made. Only those tests with a connection to the defect must be rerun, but retesting must start at the point of repair if it is before the point of failure. For example, the retest of a failed FTP process may use an existing data file instead of recreating it if up to that point everything else was OK.
SYSTEM TESTING
System testing tests all components and modules that are new, changed, affected by a change, or needed to form the complete application. The system test may require involvement of other systems but this should be minimized as much as possible to reduce the risk of externally-induced problems. Testing the interaction with other parts of the complete system comes in Integration Testing. The emphasis in system testing is validating and verifying the functional design specification and seeing how all the modules work together. For example, the system test for a new web interface that collects user input for addition to a database doesn’t need to include the database’s ETL application—processing can stop when the data is moved to the data staging area if there is one. The first system test is often a smoke test. This is an informal quick-and-dirty run through of the application’s major functions without bothering with details. The term comes from the hardware testing practice of turning on a new piece of equipment for the first time and considering it a success if it doesn’t start smoking or burst into flame. System testing requires many test runs because it entails feature by feature validation of behavior using a wide range of both normal and erroneous test inputs and data. The Test Plan is critical here because it contains descriptions of the test cases, the sequence in which the tests must be executed, and the documentation needed to be collected in each run. When an error or defect is discovered, previously executed system tests must be rerun after the repair is made to make sure that the modifications didn’t cause other problems. This will be covered in more detail in the section on regression testing.
USER ACCEPTANCE TESTING (UAT)
User Acceptance Testing is also called Beta testing, application testing, and end-user testing. Whatever you choose to call it, it’s where testing moves from the hands of the IT department into those of the business users. Software vendors often make extensive use of Beta testing, some more formally than others, because they can get users to do it for free. By the time UAT is ready to start, the IT staff has resolved in one way or another all the defects they identified. Regardless of their best efforts, though, they probably don’t find all the flaws in the application. A general rule of thumb is that no matter how bulletproof an application seems when it goes into UAT, a user somewhere can still find a sequence of commands that will produce an error.
Usability testing:
There are two major considerations when conducting usability testing. The first is to ensure that the best possible method for testing is used. Generally, the best method is to conduct a test where representative participants interact with representative scenarios. The tester collects data on the participant’s success, speed of performance, and satisfaction. The findings, including both quantitative data and qualitative observations information, are provided to designers in a test report. Using ’inspection evaluations,’ in place of well-controlled usability tests, must be done with caution. Inspection methods, such as heuristic evaluations or expert reviews, tend to generate large numbers of potential usability ’problems’ that never turn out to be actual usability problems.
The second major consideration is to ensure that an iterative approach is used. After the first test results are provided to designers, they should make changes and then have the Web site tested again. Generally, the more iterations, the better the Web site.
UI testing
is user interface testing. That is, testing how the application and the user interact. This includes how the application handles keyboard and mouse input and how it displays screen text, images, buttons, menus, dialog boxes, icons, toolbars and more.
End-to-end testing
is the process of testing transactions or business level products as they pass right through the computer systems. Thus this generally ensures that all aspects of the business are supported by the systems under test.
This ensures that the overall process flows as expected, that is that system components integrate together correctly and that the right information is passed between them.
The process may fail and there could be omissions in the process. Dependencies may not have been identified and incorrect information could be passed between system components and systems. The software may work, but the overall process does not (for example, people order goods, but they are never delivered).
This is also known as end2end testing, end 2 end testing and End-End testing.
SMOKE TESTING:
• Smoke testing originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch fire and smoke. In software industry, smoke testing is a shallow and wide approach whereby all areas of the application without getting into too deep, is tested.
• A smoke test is scripted, either using a written set of tests or an automated test
• A Smoke test is designed to touch every part of the application in a cursory way. It’s shallow and wide.
• Smoke testing is conducted to ensure whether the most crucial functions of a program are working, but not bothering with finer details. (Such as build verification).
• Smoke testing is normal health check up to a build of an application before taking it to testing in depth.
SANITY TESTING:
• A sanity test is a narrow regression test that focuses on one or a few areas of functionality. Sanity testing is usually narrow and deep.
• A sanity test is usually unscripted.
• A Sanity test is used to determine a small section of the application is still working after a minor change.
• Sanity testing is a cursory testing, it is performed whenever a cursory testing is sufficient to prove the application is functioning according to specifications. This level of testing is a subset of regression testing.
• Sanity testing is to verify whether requirements are met or not, checking all features breadth-first.
Regression Testing:
If a piece of Software is modified for any reason testing needs to be done to ensure that it works as specified and that it has not negatively impacted any functionality that it offered previously. This is known as Regression Testing.
Regression Testing attempts to verify:
- That the application works as specified even after the changes/additions/modification were made to it
- The original functionality continues to work as specified even after changes/additions/modification to the software application
- The changes/additions/modification to the software application have not introduced any new bugs
Acceptance testing
is a final stage of testing that is performed on a system prior to the system being delivered to a live environment. Systems subjected to acceptance testing might include such deliverables as a software system or a mechanical hardware system. Acceptance tests are generally performed as "black box" tests. Black box testing means that the tester uses specified inputs into the system and verifies that the resulting outputs are correct, without knowledge of the system's internal workings.
User acceptance testing (UAT), is the term used when the acceptance tests are performed by the person or persons who will be using the live system once it is delivered. If the system is being built or developed by an external supplier, this is sometimes called customer acceptance testing (CAT). The UAT or CAT acts as a final confirmation that the system is ready for go-live. A successful acceptance test at this stage may be a contractual requirement prior to the system being signed off by the client.
In some industries, such as manufacturing, factory acceptance testing (FAT), is the term usually used for the formal acceptance test stage. A factory acceptance test may be performed by the vendor or supplier of the system and observed by the client. This is often done in a very formal manner, with each factory or site acceptance test being documented in detail, and each portion of the test signed off.
Compatibility Testing:
Compatibility tests insures that the application works with differently configured systems based on what the users have or may have. When testing a web interface, this means testing for compatibility with different browsers and connection speeds.