Random testing is a black-box software testing technique where programs are tested by generating random, independent inputs. Results of the output are compared against software specifications to verify that the test output is pass or fail. Sometime this approach is useful but sometime not. So I want to know what can be the advantages and disadvantages of random testing?
Advantages: Quick. Easy to program. Routine. Exercises the main paths.
Disadvantage: Dirty. It does not think about the myriad of things that happen in real life: Comm lines breaking down, hackers active, weird inputs from users that are unthought of...
Another advantage is that in complex systems random testing could produce inputs a user would not think to try. Of course in really complex systems this testing technique is one of the few viable ones, as the brute force testing may fail (i.e. testing all the possible inputs).
On the other hand the randomness of the method makes sure that you will have not tested some inputs, some of which may be standard to a human tester, and more important to the system's operation.
The advantage is the fast testing and fast simulation patterns. The disadvantage is some biased outputs
The advantages of these test are:
- they're are done from a user’s point of view
- do not require to know programming languages or how the software has been implemented
- can be conducted by a body independent from the developers,
- can be designed as soon as the specifications are complete
Note that they can be not totally automated by opting to guided random tests.
Random testing can be instituted in a manner that creates a robust, resilient system. A major provider of network content has a mechanism called "chaos monkey" that randomly changes system parameters, drops system connections and so on.
This type of random testing supports the development and deployment of systems that are stable of a wide range of environmental conditions.
I see three types of testing, each one for different purposes and with different sets of advantages/disadvantages:
1. random testing; to explore the unknown, and the learn what to expect (domain/events/data/performance ...). This is strange when designing (you tend to study the domain first, right) but could be a good addition when the domain knowledge might be incomplete, like in case of resilience studies
2. synthetic testing; to study the correct functioning of design decisions, such as: this parameter should not be zero, you test a smaller-than-zero, zero, and greater-than-zero value. This is known as the classical equivalent class/boundary value testing method. This is typically done during alpha (factory) testing. You known the domain, you know the design decision, and you design test cases that verify these design decisions
3. performance testing. In this case, you add real data in bulk to obtain a good indication for true performance. In case this bulk data set is incomplete in that it does not (or cannot) sufficiently cover the domain dealing with, you may not obtain a good indication on resilient behavior. Adding a test method like under 1) can add this. This may be needed in case the preselected bulk data do not cover all data values/events etc. that also occur in real life conditions, but only a subset. An example is the testing of compilers: as new language features are being added to the language, the historical test data base, with test cases built up over time, may no longer be sufficient and random testing (approach 1) would quickly reveal this.
As per my opinion, te main advantage of random testing is : it is cheaper, simpler and easier. It gives shallow but wide coverage compared to targeted testing
Random testing is certainly useful and interesting, but can't do the whole job.
One of the disadvantages of random testing is that it may not discover corner cases. Imagine a function with two integer inputs - let's call them A and B - and each of them can take any value from 0 up to 1 million. It will take a random tester on average 1 million tries to find a case where A = B. That's a test case that a little domain knowledge might have suggested.
True, with some random testers, you can weight the test to try small numbers more often, or similar strategies. That's a compromise, and still some corner cases will get past it that domain knowledge might have suggested.
Don't take this as a dismissal of random testing. It is useful.
The main advantage of random testing is that it is easy to generate. However, random testing does not ensure the fair distribution of parameters-values and stable strength of coverage. Unlike the sampling strategy which provides a subset of exhaustive test in more deterministic manner.
- Badges
- Science topic
- Similar topics
- Philosophy
- Philosophy Of Science
- Reasoning