Q: What criteria do you consider for automating a test?
A: I would consider the following points to help me decide if a test should be automated:
How often does the test need to be executed? i.e. is that going to be a regression test? Sometimes the test will need to be executed once, but with a large set of data
How much time does automating this test will save me so that I can use my time in exploratory testing
How important is the test to the business; i.e. is the test scenario a typical user journey through the application
How complex is it to automate the test and how likely is it that the complexity doesn’t cause many false positives which increases results analysis time?
How likely is it that this test catches a defect?
How likely is it that a feature or functionality will break and what is the impact of it to the business? If it is high impact, then it should be automated to ensure it passes from release to release
Q: What kind of tests should NOT be automated?
A: This interview question is similar to previous question but focuses on which tests Not to be automated and left for manual testing. Possible answers can be:
Usability Testing – at times this can be an impossible task to perform by automation as the computer cannot efficiently judge if the system is of any use to its users
Tests that only need to be executed once – unless the same test needs to be executed for a large dataset then it makes sense to automate
Tests without predictable results – test automation should give us confidence in the results of the tests. If there are intermittent failures then the tests cannot be reliable and cannot be dependent on
Tests that need to be verified visually
Tests that need to be executed quickly. At first, writing an automated test takes longer. If we want a quick check, we should test manually, however, if that test is a good one which should be run regularly, then it should be automated in time
Q: What are Pros and Cons of automating tests at UI layer?
: Pros
UI automated tests execute in a way that simulates user interacting with the system. So it is very good for validating user journeys and flows
Can cover end-to-end flows that communicate with 3rd party systems
Because tests are run against the system, they can be demoed to the customer who can understand what tests are run
Can catch high severity or show stopper bugs
Can check UI functionality where it is not possible to test otherwise
: Cons
UI automated tests can be very brittle (i.e. fail due to UI changes even though functionality hasn’t changed)
Slow feedback to the team. Execution is slow as you have to wait for the system to launch and connections with 3rd party system can take a long time
Limitation on what can be checked from the UI. There are some information that are not present from the UI
Because tests are slow from UI, we can’t have a lot of tests running against the UI
Can be time consuming to construct automated test scripts for the UI
Usually have to depend on a 3rd party tool or vendor for UI testing
No comments:
Post a Comment