Test Automation for QTP
View more presentations from krayachoti.
Wednesday, November 24, 2010
Sunday, July 5, 2009
Defect Removal Efficiency
In the recent past,IT projects are spending more time in arriving at 'defect prediction', 'defect detection', defect prevention' activities as this plays a key driver for the overall QUALITY of the product / project.
Out of this, defect removal efficiency is the one which is getting more important to the project teams as this is one of the measurement to assess the quality of the project team delivery.
The below is the formula that we normally use to calculate DRE (DefectRemoval Efficiency).
DRE =
Total Number of Defects in Dev + QA
________________________________________________ * 100
Total Number of Defects in Dev + QA + Production
This gives the % of DRE for the particular project. Different people have given different perspective on how to analyze the high quality based on the results.
But tokeep this very simple and straight forward, based on the readings of many references, I prefer going as per the below standards to arrive at the High quality measurements.
SEI-CMM Level 1 - 85%
SEI-CMM Level 2 - 89%
SEI-CMM Level 3 - 91%
SEI-CMM Level 4 - 94%
SEI-CMM Level 5 - 97%
This % may vary to some extent depending on the company / project goals. Ideally the above will be the reasonable numbers.If the software / product is being used by many clients, then we need to add one more parameter based on the client numbers.
Out of this, defect removal efficiency is the one which is getting more important to the project teams as this is one of the measurement to assess the quality of the project team delivery.
The below is the formula that we normally use to calculate DRE (DefectRemoval Efficiency).
DRE =
Total Number of Defects in Dev + QA
________________________________________________ * 100
Total Number of Defects in Dev + QA + Production
This gives the % of DRE for the particular project. Different people have given different perspective on how to analyze the high quality based on the results.
But tokeep this very simple and straight forward, based on the readings of many references, I prefer going as per the below standards to arrive at the High quality measurements.
SEI-CMM Level 1 - 85%
SEI-CMM Level 2 - 89%
SEI-CMM Level 3 - 91%
SEI-CMM Level 4 - 94%
SEI-CMM Level 5 - 97%
This % may vary to some extent depending on the company / project goals. Ideally the above will be the reasonable numbers.If the software / product is being used by many clients, then we need to add one more parameter based on the client numbers.
Friday, July 3, 2009
Business Operation Testing
In today’s testing world, one of the challenges is to convert the requirements into actual functional test cases (Manual and Automation) and to assure appropriate coverage. One way of dealing with this is involving the business analysts at the beginning of the test design stage till test case execution stage. This helps in achieving more coverage in terms of testing the desired business functionalities.
There is a way that business analysts can involve in the test case design and test data management. This process is called ‘Business Operation Testing’.
There is a way that business analysts can involve in the test case design and test data management. This process is called ‘Business Operation Testing’.
Sunday, June 28, 2009
Testing as a Career !!!
"Software Testing is considered to be the tail end of the IT projects and the people involved to do testing is non-technical ones"....this is one of the oldest statement for testing. But over a period, importance of testing has drastically increased and it is embedded into the project life cycle right from the beginning. Testing paradigm has changed and testing has 2 wings, manual & automation.
With the introduction of automation testing and also various types of testing like security, data base, performance and many more types of testing one requires as much technical expertise as developers along with their domain knowledge. Out of all this, automation testing has its own space and emerging as one of the strongest drivers for ROI on testing / quality for organizations.
It will not be exaggerate to say that some companies are maintaining 1:1 ratio of developers & testers to maintain the quality of the software. Eventually testing became one of the key focus areas of technology streams of IT companies and investing a lot on this area. Now testing is a practice rather than a process / stage of a SDLC.
Testing will drive the entire team towards the goal with perfection. Now a day’s trainee who just started their career are showing interest to mold their careers into testing / automation testing. One of the key benefits of testing is, it gives the complete business cycle knowledge and also domain expertise. It will tell you why the software has been implemented and who is the exact target audience and what is the ROI of the particular application / software to the company.
Earlier UAT (User Acceptance Testing) used to happen at the client's place with the business users, who are not professionally trained testers. But this trend is changing, as business users are now getting training on the testing concepts and with the help of testing team are able to do effective UAT. Some companies are going one more step further and getting their UAT done from the third party testing team from different vendors.
However, the opportunities have been increased in testing and particular to automation testing it is even more demanding career now. People are showing more interest towards automation testing, as they will be doing the script development in the automation tool preferred scripting languages which is more or like development work. In automation testing, one has to work like a developer & tester and this combination gives a lot wider scope in terms of learning to the people who chose this stream of area.
Coming to automation, HP tools are the market leaders like QTP, QC in most of the test automation engagements.
If some one is interested to have testing as career, it is always good to learn the manual testing concepts and to learn one of the automation tools (preferably QTP) to have a smooth path for the entry.
QTP is versatile tool and it is very flexible and technology agnostic. To learn QTP, it is better to have a basic understanding of the VB script, some programming techniques, HTML & DOM object concepts. This will keep the tester profile more competent to other testers in the market. There are some online QTP courses which one can do without much effort if he/she has good understanding on QTP. It is good to have the certificate before we attend any interview, as the certificate will talk about you before you give the interview.
Things are changed a bit now ...'Testing is considered to be one of the primary focus for the IT projects and measures like automation testing being introduced into the project life cycle form the beginning to get the delivery done on time / budget with more QUALITY'....
With the introduction of automation testing and also various types of testing like security, data base, performance and many more types of testing one requires as much technical expertise as developers along with their domain knowledge. Out of all this, automation testing has its own space and emerging as one of the strongest drivers for ROI on testing / quality for organizations.
It will not be exaggerate to say that some companies are maintaining 1:1 ratio of developers & testers to maintain the quality of the software. Eventually testing became one of the key focus areas of technology streams of IT companies and investing a lot on this area. Now testing is a practice rather than a process / stage of a SDLC.
Testing will drive the entire team towards the goal with perfection. Now a day’s trainee who just started their career are showing interest to mold their careers into testing / automation testing. One of the key benefits of testing is, it gives the complete business cycle knowledge and also domain expertise. It will tell you why the software has been implemented and who is the exact target audience and what is the ROI of the particular application / software to the company.
Earlier UAT (User Acceptance Testing) used to happen at the client's place with the business users, who are not professionally trained testers. But this trend is changing, as business users are now getting training on the testing concepts and with the help of testing team are able to do effective UAT. Some companies are going one more step further and getting their UAT done from the third party testing team from different vendors.
However, the opportunities have been increased in testing and particular to automation testing it is even more demanding career now. People are showing more interest towards automation testing, as they will be doing the script development in the automation tool preferred scripting languages which is more or like development work. In automation testing, one has to work like a developer & tester and this combination gives a lot wider scope in terms of learning to the people who chose this stream of area.
Coming to automation, HP tools are the market leaders like QTP, QC in most of the test automation engagements.
If some one is interested to have testing as career, it is always good to learn the manual testing concepts and to learn one of the automation tools (preferably QTP) to have a smooth path for the entry.
QTP is versatile tool and it is very flexible and technology agnostic. To learn QTP, it is better to have a basic understanding of the VB script, some programming techniques, HTML & DOM object concepts. This will keep the tester profile more competent to other testers in the market. There are some online QTP courses which one can do without much effort if he/she has good understanding on QTP. It is good to have the certificate before we attend any interview, as the certificate will talk about you before you give the interview.
Things are changed a bit now ...'Testing is considered to be one of the primary focus for the IT projects and measures like automation testing being introduced into the project life cycle form the beginning to get the delivery done on time / budget with more QUALITY'....
Selenium Recommendation
Selenium
Selenium enables you to easily test web applications that contain rich, client-side, interactive content. For example, you can use Selenium when testing an AJAX application such as an application written with Microsoft Atlas.
Selenium enables you to record a browser session. For example, you could record the series of actions required to add a product to a shopping cart or register at a website.
You can use Selenium to record each significant client-side action that a user performs in a browser such as clicking buttons and links and entering text into input fields. After you record a test, you can run the test with any browser. For example, you can execute the same test with Internet Explorer, Firefox, and Opera to check that your application is cross-browser compatible.
When you record each step in a test, you can include assertions that the browser has a certain property. For example, you can include an assertion that a particular string of text is displayed by the browser or an assertion that the browser is displaying a particular title in its title bar. The ability to create assertions provides you with the ability to create tests.
Selenium is a good tool to use when building a variety of different types of automated tests. For example, you can use Selenium to create regression tests. Whenever you modify your application, you can run a suite of Selenium tests to verify that your modifications have not broken code that previously functioned.
Downloading Selenium
Selenium is an open-source project (released under the Apache license) created by the developers at Thought Works. You can download Selenium from the OpenQA website at http://www.OpenQA.com/Selenium.
Types of Selenium Tools / Products:
There are three different Selenium Tools:
Selenium IDE — enables you to record a browser session. The Selenium IDE is a Firefox extension so it only works with Firefox. However, tests that you create with the Selenium IDE can be run on other browsers including Internet Explorer and Opera.
Selenium Core — enables you to run Selenium tests. You can use the Selenium Core library to execute a suite of tests from a browser.
Selenium RC — enables you to remotely control a browser. Includes a C# wrapper for Selenium so that you can remotely control a browser from a C# application. Selenium RC is especially useful when used with unit testing tools such as nUnit.
Language of the Tool:
Selenium IDE only supports testing in "Selenese", a simple scripting language.
· Selenium Remote Control allows to write the tests in any programming language, including Java, .NET, Perl, Python and Ruby.
Recording a Browser Test
The easiest way to understand how Selenium works is to download the Selenium IDE. You’ll need to install Firefox before you can use the Selenium IDE since the Selenium IDE is implemented as a Firefox extension.
After you install the Selenium IDE, you are ready to start recording tests. Launch Firefox and navigate to your favorite website (for example, Superexpert.com). Open the Firefox Tools menu. You should see a menu item labeled Selenium IDE. Select this menu item to open the Selenium IDE (see Figure 1). After you open the Selenium IDE, your browser session is recorded automatically
Process to record a action:
Navigate around the website. Fill out a form or two. If you take a look at the Selenium IDE after performing an action, you’ll notice that your action has been recorded in an HTML table.
You can also record assertions at any time. For example, you might want to record an assertion that a page has a certain title or that the page contains certain text. To create an assertion, right-click in Firefox and pick a command. For example, you can select the assertTitle command to assert that a particular title is being displayed in the browser
After you are done recording a test, you can click the Record button (the big red circle) to turn off recording.
Now that you have created a test, you can run the test. You can run the test directly from the Selenium IDE within Firefox by selecting among the Run, Walk, and Step radio buttons. When you run a test, the test is executed from start to finish without pause. When you walk a test, the test is executed from start to finish with a pause between each step. Finally, when you step through a test, you must click the Step button to advance to each new step in the test.
As each command in the test executes, the cell that contains the command turns either red or green. If an assertion fails, then the cell turns red. Otherwise, if the step succeeds, the cell turns green.
By default, the Selenium IDE records tests in an HTML table. As an alternative, you can record the steps of a test as a series of C# commands. Select the menu option Options, Format, C#. If you record a C# test, then you can run the test using Selenium RC.
Using Selenium Test Runner
The Selenium IDE only works with Firefox. If you want to automatically record a test, then you must do it with Firefox. However, you can run a test using any modern web browser.
If you want to run a test outside of the Selenium IDE, then you need to install Selenium Core. After downloading Selenium Core and extracting the files, copy the Core subfolder into your Inetpub\wwwroot folder. The Core folder includes the TestRunner.html page. Rename the Core folder to Selenium.
After you have added the Selenium folder to your wwwroot folder, you can request the TestRunner.html page from any Web browser.
Open Internet Explorer and enter the following page request in the address bar:
http://localhost/Selenium/TestRunner.html
The Selenium TestRunner page enables you to execute any suite of Selenium tests. You select a suite of tests to run with the left-hand column.
A test suite is simply a list of Selenium tests. You create a test suite by creating an HTML page with a list of links to individual tests. For example, the following page is a test suite that contains four tests.
The Selenium Core download includes a Tests folder. You can copy this folder into your Inetpub\wwwroot folder. When you open TestRunner.html, select either TestSuite.html or ShortTestSuite.html as the Test Suite. After you select a test suite, you can run the tests by clicking the All button in the right-hand side of the page.
TestRunner.html can only run tests against web applications on the local server. You won’t get it to work against a remote website such as Google.com. TestRunner.html is blocked from working with remote sites because of cross-domain scripting security restrictions.
If you need to execute tests against a remote website, then you have three options. First, you can run the test within Selenium IDE. Since Selenium IDE is implemented as a Firefox extension, Selenium IDE is not subject to the same security restrictions as a normal web page. The disadvantage of Selenium IDE is that it only works with Firefox and not with other browsers such as Internet Explorer.
Second, you can run the test against a remote website by using the HTML Application (HTA) version of TestRunner. The TestRunner.hta page is located in the same folder as the TestRunner.html page. HTML Applications, like Firefox extensions, are not subject to the same security restrictions as a normal web page. The disadvantage of using the HTA version of TestRunner is that it only works with Internet Explorer and not other Browsers such as Firefox or Opera.
The final, and best, option for executing tests against a remote website is to use Selenium RC. Selenium RC can be used to control any modern browser. It also works fine with remote domains. Selenium RC is a separate download from the Selenium website.
Using nSelenium
To use the nSelenium application, you must create two folders in your Inetpub\wwwroot folder: a Selenium folder and a nSelenium folder. The Selenium folder should be the Core folder from the Selenium Core download. The nSelenium should contain the nSelenium application itself.
You can add new tests to nSelenium through a form interface (see Figure 5). Record the test with Selenium IDE and then past the test into nSelenium. Internally, the nSelenium application stores the tests in a Microsoft SQL Express database.
After you add one or more tests to nSelenium, you can run the tests one at a time or you can run all of the tests at the same time. Either click the Run link that appears next to each test or click the Run All Tests link that appears at the bottom of the page. When you run the tests, the test results are automatically stored in the database. You can see all of the past results by clicking the Results link that appears next to each test.
Selenium Tool Advantages:
Selenium enables you to easily test web applications that contain rich, client-side, interactive content. For example, you can use Selenium when testing an AJAX application such as an application written with Microsoft Atlas.
Selenium enables you to record a browser session. For example, you could record the series of actions required to add a product to a shopping cart or register at a website.
You can use Selenium to record each significant client-side action that a user performs in a browser such as clicking buttons and links and entering text into input fields. After you record a test, you can run the test with any browser. For example, you can execute the same test with Internet Explorer, Firefox, and Opera to check that your application is cross-browser compatible.
When you record each step in a test, you can include assertions that the browser has a certain property. For example, you can include an assertion that a particular string of text is displayed by the browser or an assertion that the browser is displaying a particular title in its title bar. The ability to create assertions provides you with the ability to create tests.
Selenium is a good tool to use when building a variety of different types of automated tests. For example, you can use Selenium to create regression tests. Whenever you modify your application, you can run a suite of Selenium tests to verify that your modifications have not broken code that previously functioned.
Downloading Selenium
Selenium is an open-source project (released under the Apache license) created by the developers at Thought Works. You can download Selenium from the OpenQA website at http://www.OpenQA.com/Selenium.
Types of Selenium Tools / Products:
There are three different Selenium Tools:
Selenium IDE — enables you to record a browser session. The Selenium IDE is a Firefox extension so it only works with Firefox. However, tests that you create with the Selenium IDE can be run on other browsers including Internet Explorer and Opera.
Selenium Core — enables you to run Selenium tests. You can use the Selenium Core library to execute a suite of tests from a browser.
Selenium RC — enables you to remotely control a browser. Includes a C# wrapper for Selenium so that you can remotely control a browser from a C# application. Selenium RC is especially useful when used with unit testing tools such as nUnit.
Language of the Tool:
Selenium IDE only supports testing in "Selenese", a simple scripting language.
· Selenium Remote Control allows to write the tests in any programming language, including Java, .NET, Perl, Python and Ruby.
Recording a Browser Test
The easiest way to understand how Selenium works is to download the Selenium IDE. You’ll need to install Firefox before you can use the Selenium IDE since the Selenium IDE is implemented as a Firefox extension.
After you install the Selenium IDE, you are ready to start recording tests. Launch Firefox and navigate to your favorite website (for example, Superexpert.com). Open the Firefox Tools menu. You should see a menu item labeled Selenium IDE. Select this menu item to open the Selenium IDE (see Figure 1). After you open the Selenium IDE, your browser session is recorded automatically
Process to record a action:
Navigate around the website. Fill out a form or two. If you take a look at the Selenium IDE after performing an action, you’ll notice that your action has been recorded in an HTML table.
You can also record assertions at any time. For example, you might want to record an assertion that a page has a certain title or that the page contains certain text. To create an assertion, right-click in Firefox and pick a command. For example, you can select the assertTitle command to assert that a particular title is being displayed in the browser
After you are done recording a test, you can click the Record button (the big red circle) to turn off recording.
Now that you have created a test, you can run the test. You can run the test directly from the Selenium IDE within Firefox by selecting among the Run, Walk, and Step radio buttons. When you run a test, the test is executed from start to finish without pause. When you walk a test, the test is executed from start to finish with a pause between each step. Finally, when you step through a test, you must click the Step button to advance to each new step in the test.
As each command in the test executes, the cell that contains the command turns either red or green. If an assertion fails, then the cell turns red. Otherwise, if the step succeeds, the cell turns green.
By default, the Selenium IDE records tests in an HTML table. As an alternative, you can record the steps of a test as a series of C# commands. Select the menu option Options, Format, C#. If you record a C# test, then you can run the test using Selenium RC.
Using Selenium Test Runner
The Selenium IDE only works with Firefox. If you want to automatically record a test, then you must do it with Firefox. However, you can run a test using any modern web browser.
If you want to run a test outside of the Selenium IDE, then you need to install Selenium Core. After downloading Selenium Core and extracting the files, copy the Core subfolder into your Inetpub\wwwroot folder. The Core folder includes the TestRunner.html page. Rename the Core folder to Selenium.
After you have added the Selenium folder to your wwwroot folder, you can request the TestRunner.html page from any Web browser.
Open Internet Explorer and enter the following page request in the address bar:
http://localhost/Selenium/TestRunner.html
The Selenium TestRunner page enables you to execute any suite of Selenium tests. You select a suite of tests to run with the left-hand column.
A test suite is simply a list of Selenium tests. You create a test suite by creating an HTML page with a list of links to individual tests. For example, the following page is a test suite that contains four tests.
| Test Suite |
| TestOpen |
| TestClick |
| TestType |
| TestSelect |
The Selenium Core download includes a Tests folder. You can copy this folder into your Inetpub\wwwroot folder. When you open TestRunner.html, select either TestSuite.html or ShortTestSuite.html as the Test Suite. After you select a test suite, you can run the tests by clicking the All button in the right-hand side of the page.
TestRunner.html can only run tests against web applications on the local server. You won’t get it to work against a remote website such as Google.com. TestRunner.html is blocked from working with remote sites because of cross-domain scripting security restrictions.
If you need to execute tests against a remote website, then you have three options. First, you can run the test within Selenium IDE. Since Selenium IDE is implemented as a Firefox extension, Selenium IDE is not subject to the same security restrictions as a normal web page. The disadvantage of Selenium IDE is that it only works with Firefox and not with other browsers such as Internet Explorer.
Second, you can run the test against a remote website by using the HTML Application (HTA) version of TestRunner. The TestRunner.hta page is located in the same folder as the TestRunner.html page. HTML Applications, like Firefox extensions, are not subject to the same security restrictions as a normal web page. The disadvantage of using the HTA version of TestRunner is that it only works with Internet Explorer and not other Browsers such as Firefox or Opera.
The final, and best, option for executing tests against a remote website is to use Selenium RC. Selenium RC can be used to control any modern browser. It also works fine with remote domains. Selenium RC is a separate download from the Selenium website.
Using nSelenium
To use the nSelenium application, you must create two folders in your Inetpub\wwwroot folder: a Selenium folder and a nSelenium folder. The Selenium folder should be the Core folder from the Selenium Core download. The nSelenium should contain the nSelenium application itself.
You can add new tests to nSelenium through a form interface (see Figure 5). Record the test with Selenium IDE and then past the test into nSelenium. Internally, the nSelenium application stores the tests in a Microsoft SQL Express database.
After you add one or more tests to nSelenium, you can run the tests one at a time or you can run all of the tests at the same time. Either click the Run link that appears next to each test or click the Run All Tests link that appears at the bottom of the page. When you run the tests, the test results are automatically stored in the database. You can see all of the past results by clicking the Results link that appears next to each test.
Selenium Tool Advantages:
- The core is written in java script so it runs directly in a browser, just as real users do
- Supported on OS X, Linux, and Windows
- Very simple to learn and use.
- Open Source (Apache License 2.0)
- Selenium Remote Control
- Selenese has a number of strict limitations: Selenium doesn’t support conditional statements (Ex: "if" statement), and loops (Ex: "for" statements). This can make writing complicated tests difficult or even impossible.
- Selenium IDE cannot be installed in any browser other than Mozilla fire fox.
- Selenium is not meant for unit testing.
- Minor Issue with SSL: Selenium can't switch properly between http and https pages.
- File Input type not supported: User can’t upload a file in Selenium.
Subscribe to:
Posts (Atom)
