Unit Test Automation with TestNG

14 min readSep 10, 2021

--

Why do we need Test Automation?

When we are going through hundreds of test cases manually, it will waste so much time, and most of the time meeting the deadlines with manual testing is difficult. So that Test automation is important for QA engineers and automation engineers.

When it comes to Test automation there are 3 different levels of testing.

Unit Testing
API Testing
End to End testing(UI Testing)

we call this an automation pyramid.

Today we are going to talk about unit level testing, for this we need to know about unit-level frameworks.

Unit Level testing

We consider the unit test as the smallest test.

Unit testing is a software development process in which the smallest testable parts of an application called units, are individually and independently tested.
Unit testing can be done manually but is often automated.

Think that you have written logic for validation of Input data of a form

We can do this manually but every time when you are doing a change in the code, testing manually is difficult and time-consuming.

if you have automated this test, you can run the script and it will tell whether the test is passed or not. so this will increase your efficiency.

Any programming language will have its own unit testing framework.

for java by default unit testing framework will be Junit, similarly for .Net it's Nunit, due to the limitations of Junit in java there are other extensions of Junit which is known as TestNG (Test Next Generation). for angular,unit test framework is going to be jasmine. for C++, there are many frameworks and the most popular one is the google test, for MYSQL and any DB also have unit testing frameworks.

so in our program, we are going to focus on TestNG.

There are some drawbacks in Junit4 in order to solve those issues, TestNG was introduced .and TestNG is giving the following functionalities than Junit4.

Group testing
Parameterized object testing.
Dependency Testing

Now let's start the exercises.

Here first make a maven project in Inteliji IDE.After making a maven Project Your project Structure is visible as below.

Setting up TestNG

Here First go to the pom.xml file and the maven should be maven 3.

Here you can use this site and get maven dependency copied.

https://mvnrepository.com/artifact/org.testng/testng/7.3.0

img2-site with maven dependencies for TestNG

then paste highlighted dependency in the pom.xml file of the project as below.

then after installing it, the external libraries should look as follows.

Now let's start to study the annotations We can use for unit Testing in TextNG.

Here in img6, Under the “test” folder in the java package, we can create a folder or java classes for Testing. Here I will create a Folder call the “annotations.test” folder and let's create a java class with the name “Test1” inside that package as well.

Let's start studying annotations of TestNG.

@Test annotation

Normally in a java class, we need to have a main class to run any method as below. You even get the run icon to run the main method and for other methods, you don’t. but when we add @Test annotation for other methods you will get a run button for each individual method .so that you can run each and every method individually. Make sure when you are using @Test annotation you should be careful about the library you are importing it is not “org.junit” library but “org.testng.annotations” library.

As below you will get a run icon for each method that you have used @Test annotation.

img8 -(run button appears on the right side with the presence of @Test annotation)

The behaviour of @BeforeMethod,@Aftermethod,@BeforeClass and @Afterclass annotations

When we consider the below code.

img9-(Example with @Test,@Beforemethod and @Aftermethod annotations)

When we run the “test1” method only, the output will be as below.

img10-Output of img9 when we run only the test1 method

When we run the whole class then the output will look as below.

img11-output of img9 when we run Test1 class wise

So as you can see it will execute the method with @Beforemethod annotation before each and every method with @Test annotation.

and the method with @Aftermethod annotation will be executed at the end of each and every test method.

Then for the same code let's add methods with @Afterclass and @Beforeclass annotations as below.

img12-@Afterclass and @BeforeClass annotations

Then the output is as below when we run the whole class.

Here you can see. once the class is loaded if there is a method with @Beforeclass it will be executed first. and it will be executed only once no matter how many test methods you have, The methods with @Beforemethod annotation will execute prior to each test method.

when you look at @Beforeclass and @Beforemethod, it is similar to use them to define preconditions .whatever the preconditions you need to give you can define them in methods with @Beforeclass or @Beforemethod annotations.

we normally write the setup process in the @Beforeclass method.

for example, let’s think we want to test whether CRUD operations of a form are working properly. normally, there are preconditions like going to the browser and log in. when we are using @Beforemethod, for each test, The preconditions will run. that means if there are 5 tests when we run them predefine conditions inside the method with @Beforemethod annotation will get executed 5 times. So according to this example, we will open the browser and login in 5 times. but if we want to open a browser and log in only once, that predefined condition can be written in a method with @Beforeclass annotation as it will get executed only once through the whole process of the Test class.

all these steps are based on the application architecture and the execution time and efficiency you expect that your application to have.

@Afterclass annotation will get executed only once at the end of all the test classes. It is the opposite of @Beforeclass annotation.

Priorities

when you are looking into the above program, you may choose the below output.

Login
Create
Edit 
Delete
Logout

but the above answer is wrong. Here the TestNG is prioritizing the methods by the alphabetical order of the Test method name. So the correct output of the above program is as follows.

So, How can we prioritize one method over another as we want?.

The answer is setting the execution order.

Setting the execution order

for this, we can use different attributes with @Test annotation to define the order it needs to get executed.

Priority attribute

When we add priority to each Test, it will get executed according to that order. and if we have the same priority for two test methods, then alphabetical order will be considered. Evey Test method is having priority values of 0 in default.

img15- with attribute priority in @Test annotation.

for the above example, the output is as follows.

if you have few Tests in a test class, it will be easy to handle methods with the use of priorities but when it comes to a higher number of test methods within one class it would be difficult to handle .so when it comes to TestNG rather than setting priorities you can set dependencies. Now, let's focus on that.

dependsOnMethod Attribute

Here what we do is defining the name of the methods that a specific Test rely on. It will be defined with the use of “dependsOnMethod” attribute.

Here as You can see We have written

@Test(dependOnMethods = “testLogin”)

in the testCreate() method as it depend on testLogin() method .

Here in the above example

testCreate() method depend on testLogin()
testEdit() method depend on testCreate()
testDelete() method depend on testEdit()
testLogout() method depend on testDelete()

We even can put two dependencies as below

testDelete() method depend on testEdit() and testCreate methods.

Img18- class with methods having more than 1 dependency methods in dependsOnMethod attribute

Here, the output will be the same as the img17.

Here the test methods get executed when the methods that they depend on got executed.

Up to now, when we are executing, we did run the test method individually or we did run the whole class. When we have multiple classes, How can we execute them?

For this, there are different methods in different IDE’s.

Select and Run

img19-way to select and run few classes at the sometimes .

Creating a Test suite

If you are going to run a set of test classes in a Linux console or CICD environment. You need a test suite which is a collection of test classes.

In testNG, you need to make an XML file in the root folder. You can give any name to that file. here I give as “Testsuite1”. after that include the below XML code.

Make sure to create this XML file in the root folder .at the same level as the pom.xml file.

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >

<suite name="Suite1" verbose="1" >
     <!---package name that testclasses exists-->
    <test name="Testpackagename" >
        <classes>           <!---Here define the classes you need to run 
   for more understanding follow the below example -->           
            <class name="Testpackakename.Testclassname" />
          
        </classes>
    </test>


</suite>

according to My case. I have defined it Like this.

Here I have two packages and the one I am using here is annotation.testng.

Here, I use three test classes. Test1,TestNGtest1 and TestNGtest2 as in the above example.

Then you can right-click on the XML file and run it. incase if it is not showing the run button. then You can go to maven and inside lifecycle select clean and press the run button and try again.

Here, in my case, the output will look as follows

img21-Output of img 20 and the tests and the test classes which got executed by test suite

Why having a testsuite.XML file important to us?

You may have to test different parts of the project, each part may have multiple classes. Here if you don’t have an IDE but only java and maven still you can run the tests with the use of the test suite in the command line.

Grouping test cases

There are times that we need to group the test methods based on the features of applications and run .

We also can group Test methods based on criticality. It is known as smoke tests. (build verification tests)

Grouping test methods is a best practice in test automation as it will increase efficiency. If you select and run each test method manually, It will be a waste of time.

Now let's try to do a small example to understand the grouping.

Here first we make a package inside the java file of the test folder with the name of “groups” then make few classes inside that folder as below.

img22-group folder and group tests classes

img23-class and method with the group test method

Here as in the above metrics, for each Test class, there are 3 test methods.

grouptest1,grouptest2 and grouptest3 are the Test classes.

6 test methods are having groups. some of them belong to both groups(reg and smoke). rest are not having any group.

We are grouping the test methods as below with the use of “groups” attribute of @Test annotation.

img24- all the group classes and all of its methods.

now let's change create the TestSuite1.xml file including the below XML code.

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >

<suite name="Suite1" verbose="1" >
    <test name="groups" >
        <groups>
        <run>
   <!--Here you can include or exclude the groups that you allwant->        <include name="smoke"/>
        </run>
        </groups>
        <classes>
            <class name="groups.groupstest1" />
            <class name="groups.groupstest2" />
            <class name="groups.groupstest3" />
        </classes>
    </test>


</suite>

In this above test suite, I have included the “smoke” group inorder to run all the test methods of the “smoke” group. for the above test suite Output is as follows.

img25-output of img24 and TestSuite1.xml with include “smoke ” group

Here, when a Test method belongs to two groups (“reg” and “smoke”), those test results also get printed.

if you only want to run the classes which belong to “smoke” group only, you can exclude “reg” as below.

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >

<suite name="Suite1" verbose="1" >
    <test name="groups" >
        <groups>
        <run>
            <include name="smoke"/>
            <exclude name="reg"/>
        </run>
        </groups>
        <classes>
            <class name="groups.groupstest1" />
            <class name="groups.groupstest2" />
            <class name="groups.groupstest3" />
        </classes>
    </test>


</suite>

OUTPUT

grouping is a unique function of TestNG.

Assert Command

What is the difference between Assert and Verify?

Both Assert and Verify statements are used in the test suites for adding validations to the test methods. Testing frameworks like TestNG and JUnit are used with Selenium to provide assertions.
Coming to the difference between Assert and Verify commands. In the case of the “Assert” command, as soon as the validation fails the execution of that particular test method is stopped. Following that the test method is marked as failed.
Whereas, in the case of “Verify”, the test method continues execution even after the failure of an assertion statement. Although the test method will still be marked as failed but the remaining statements of the test method will be executed normally.
In TestNG, the Verify functionality is provided by means of the Soft Assertions or using SoftAssert class.

Here, We are focusing on Assertion which is also known as Hard Assertion.

Let’s go through some functions of Assertion.

assertEquals()

Here we are checking whether the Actual value that we get through the function is similar to the expected value. If the actual values and the expected values are not similar there will be an error.

now let's create another package called assertionTest in the test folder under the java package. then make a java class called “testngAssertion” and Generate the “testAssertEquals()” method with @Test annotation as in the img27.

here the ActualV variable gives the output of the logic ,

ActualV = “Assertion”

Expected value = “Asserton”

Since Actual Value and expected value are not the same Output will be an error.

img27-program and output of using hard assertion with the wrong expected value.

As in the img27 figure, We can give a customized failed message. when the test fails that error message will be shown. Here In the output, It will also show the line of code where the error occurs. when the actual value and expected value is the same It will not give any message.

img28-program and output of using hard assertion with correct expected value.

assertNotEquals()

Here, if the actual value is equal to the expected value, it gives the error.It is the opposite of assertEquals().

img29-assertion with asserNotEquals() and when the expected value and the actual values are the same

if the actual and expected value is not equal to each other it will not give an error.

img30-assertion with asserNotEquals() and when the expected value and the actual values are different

AssertTrue()

Here the expected result should be always True to pass the test. if the actual result is false. It will give an error.

AssertFalse()

Here It is the opposite function of the AssertEqualTrue().

Like this there are List is assertion functions that we can use. Normally in IDE’s, it will show the list of assertion functions automatically. you can select the suitable assertion function according to your preference.

Now, let’s go through few simple examples of unit testing

Unit test for Summation function

generate a java class and write a sum function as below.

img33-creating the ArithmaticOperation class with sum function inside as logic

There are some best practices we follow while naming Test classes and Test Methods.

Test classes always need to name with the class name followed by the “Test” Suffix.

Eg: Class name: ArithmeticOperator

Test Class name: ArithmeticOperatorTest

When it comes to a method It will define as a “test” prefix followed by the name of the method.

Eg: Method name: sum

Test Method name: testSum

You can write the Test with a setup function to call ArithmeticOperators object and call the relevant function(sum()) from the logic class.

then the Test class and the method will be written as in img34.

img34-creating ArithmaticTest class with relevant test functions(creating a test for the img33 logic )

Unit test for Validating Age

When we think about a small example like validating age then we can use techniques like boundary value analysis. Testing with only one test data point will not be enough to identify the bugs. Let’s consider the age of a human is from 1 to 200. we can use a minimum of four test cases to see whether the function is getting validated the age correctly as below.

According to the boundary value analysis technique, We can get 4 test data if the range is from( 1–200)

Age = 1(min)
Age=2(min+1)
Age=200(max)
Age=199(max-1)

You can write the logic as in img35.

img35-PersonalDeatils class with sum function as the logic

We can write the TestClass and test method for the above img35 logic as in img36. and when we run the test it will give the output in the img36.

Data-Driven Test

Data-driven Tests mean running the same test over and over again for different data combinations. We can run a single test for 4 different data combinations here.

Here we can use @DataProvider annotation in getData() function to provide inputs to the testAge() method, without defining the assertTrue() function for each datapoint inside the testAge() method.

Here testAge() with @Test annotation will define the name of the DataProvider function with the use of “dataProvider” attribute as in the img37.

img37-PerdonalDetailsTest class and the testAge method with @DataProvider annotation

What is the benefit of using data-driven testing?

Let's give an incorrect input and see how the output will look like. Here I have given 1990 instead of 199 to the Test program in img35.

then it will give the output as in img38.

img38-Output of img36 when we give incorrect input:1990

In img38, It will stop the process in the 3rd input data point. so that we have no clue whether the rest of the inputs will pass the test.

when it comes to data-driven test methods, it will run until the last input data point. At the end of the Test, we would know which input data points got passed through the test and which didn’t. So using DataProvider will save our time as we can get the results or bugs by running the test in one shot.