Setting up Selenium grid

In this post, we will learn how to setup Selenium grid.

What you need?

  • One or multiple machines,
  • All machines should have JAVA JDK setup (open JDK also will do)
  • selenium-server-standalone jar file
  • Chrome browser and corresponding version chrome driver
  • Selenium 3.x in your local machine from where you are going to execute the selenium

Hub Setup:

  • Login to the machine that you are going to setup as Hub (a.k.a Master) machine.
    • You can setup Hub in your local machine itself, from which you are going to run the script
    • or you can setup the Hub in separate machine.
  • Create a folder and place the selenium-server-standalone jar file inside it
  • Create a json file by name “hubconfig.json” as shown below and save it in same folder
  • create a bat file as shown below and save it in same folder

Hub configuration json

In the below Json, you have to update the “IP” as the id address of the machine (where the HUB is being setup)

{
"port": 4444,
"newSessionWaitTimeout": -1,
"servlets" : [],
"withoutServlets": [],
"custom": {},
"capabilityMatcher": "org.openqa.grid.internal.utils.DefaultCapabilityMatcher",
"throwOnCapabilityNotPresent": true,
"cleanUpCycle": 5000,
"role": "hub",
"host":"ip",
"debug": false,
"browserTimeout": 500000,
"timeout": 500000
}

startHub.bat

java -jar selenium-server-standalone.jar -role hub -hubConfig hubconfig.json
pause

That is it! you are all set. Double click on the bat file to start the Hub server. It is now ready to accept connection from a slave machine

Node (a.k.a Slave) setup:

  • Create a folder and place the selenium-server-standalone jar file inside it
    • Note: Make sure the version of this jar file is same as that used in Hub setup.
    • otherwise you will get weird errors while starting the node. check the detailed post.
  • Now create a JSON file as shown below and save it as “node.json” in same folder
  • Finally create the startNode.bat file and save it in same folder

Node Configuration JSON

  • “port” value should be the free port number in the Node machine
  • “host” will be the ip of the node machine
  • “hubHost” ip of the Hub machine as given in hubconfig.json
  • “hubPort” port of the hub machine as given in hubconfig.json
  • “applicationName” is optional.
    • If used, give a unique value for each node machine.
    • we can force execution on a particular node by passing the “applicationName” value at the time of initializing the remote web driver.
    • This way, instead of Hub controlling which node will run the test case, we can control it ourselves. we will not be at mercy of Hub machine.
    • This feature will be helpful when you know that a specific node is having a configuration, that is required to run specific test case.
{
  "capabilities":[
    {
      "platform": "WINDOWS",
      "browserName": "firefox",
      "maxInstances": 5,
      "seleniumProtocol": "WebDriver",
      "cleanSession": true,
      "applicationName": "Node8"
    },{
      "platform": "WINDOWS",
        "browserName": "chrome",
        "maxInstances": 5,
      "seleniumProtocol": "WebDriver",
      "cleanSession": true,
      "applicationName": "Node8"
    },{
      "platform": "WINDOWS",
      "browserName": "internet explorer",
      "maxInstances": 5,
      "seleniumProtocol": "WebDriver",
      "cleanSession": true,
      "applicationName": "Node8"
    }
  ],
    "maxSession":10,
    "port":5557,
    "host":"Node_ip",
    "register":true,
    "registerCycle": 5000,
    "hubPort":80,
    "hubHost":"Hub_ip",
    "nodeTimeout":120,
    "nodePolling":2000
}

startNode.bat

java -Dwebdriver.gecko.driver="C:\Selenium\Selenium Node Setup\geckodriver.exe" -Dwebdriver.chrome.driver="C:\Selenium\Selenium Node Setup\chromedriver.exe" -Dwebdriver.ie.driver="C:\Selenium\Selenium Node Setup\IEDriverServer.exe" -jar "C:\Selenium\Selenium Node Setup\selenium-server-standalone.jar" -role node -nodeConfig "C:\Selenium\Selenium Node Setup\node.json"
pause
  • it does not matter if you have firefox and IE driver or not, to create this bat script
  • But do ensure that paths are correct for the jar file
  • ensure the name of json file is “node.json” else update it accordingly in bat file

That’s it! you are all set. Make sure Hub is up and running. Go ahead and double click on startNode.bat file. It will start the node server on that machine and attach it to the Hub machine.

you can verify success message in command window. In Hub machine, you can see that the message gets logged in command window.

From your local machine (that is connected to same network as where you have setup the grid), launch chrome browser and navigate to this url(replace Hub_ip with the ip of Hub machine):

http://Hub_ip:88/grid/console

You will be able to see all the node machines connected to the Hub machine.

Execution

To run the selenium test on grid machine, you will be using remote webdriver client as shown below

WebDriver driver = new RemoteWebDriver(new URL("http://Hub_ip:88/wd/hub"));
driver.get("http://www.google.com");

if you want to run on specific node machine as i mentioned earlier, you will have to use the desired capabilities

desiredCapabilities.setCapability("applicationName","Node8");
WebDriver driver = new RemoteWebDriver(new URL("http://Hub_ip:88/wd/hub"),desiredCapabilities);
driver.get("http://www.google.com");

you can replace “Node8” with the name you specified in node.json file.

Pro tips:

  • Leveraging Windows Task Scheduler
  • If you don’t want to trouble yourself starting the Node manually each time you restart the machine, consider creating a windows task scheduler to trigger the bat file each time during restart.
  • You can create a task to even shut down the chrome browser/ chrome driver once a week (Sunday morning?), such that any stray browser instance that is not killed automatically (if test is stopped mid way manually due to which driver.quit() is not called ) will be killed by the task scheduler.
  • Reseting the Grid:
  • If you want to reset the grid for some reason (if max browser instances limit is reached on nodes), all you need to do is stop and start the hub.
  • You don’t need to restart each of the nodes separately (i’m not talking about restarting machine)
  • You can take sweet time to bring up the Hub. Nodes will keep polling till the point the hub is up. As soon as the hub is started, Nodes will auto connect to the hub.

That’s it in this post. Hope you all found it useful. Thanks.

Image handling with Java and extracting text from Image

These days many of the web apps makes use of Canvas objects, to display images in the app. These Canvas objects contain text data(dollar amount, texts etc..) that we may want to validate against source of truth. This blog we will discuss one of the ideas to deal with such situation.

  • Capture the screenshot of the page
  • Crop the image to remove unwanted part
  • Use the Tesseract to read text from image

So let’s get started.

first, we will see how we can crop the image:

BufferedImage original_Image = null;
File orig_Image_File = new File(originalImageFilePath); //file path ex: C://Images//logo.jpg   
original_Image =  ImageIO.read(orig_Image_File);

//Crop the image
File modified_Image_File = new File(modifiedImageFilePath); //save modified image in different file.
ImageIO.write(cropImage(original_Image), "jpg", modified_Image_File);


/**
*function to crop image
**/
 private BufferedImage cropImage(BufferedImage src) {
        BufferedImage dest = src.getSubimage(250, 80,300, 150);
        
        return dest; 
 }

If you notice, there are four parameters being passed to the getSubimage function. Lets see the importance of each of those numbers in the image below

Let me rewrite the function as: getSubimage(A,B,C,D). Notice the placement of these 4 points A,B,C and D on the image. if you increase value of A, the start point moves further Right. Similarly C. if you decrease the value, it moves backward (left).

If you Increase B, the top point moves down. Same with D. If we decrease the value of these points, it moves backward.

How it works?

  • For above image, Consider that total width, w=1200 and total height h = 300
  • lets specify A = 50 and C= 100
    • i.e A starts at 50th Centimeter (cm) of image
    • So c is calculated from A. i.e A+100cm
    • i.e C will not calculate from the origin point of image (0).

Points to Note:

  • Negative value is not accepted. So you need to adjust A, C or B,D in order to get the right image cropped.
  • A+C should be lesser than or equal to total width
  • similarly, B+D should be lesser than or equal to total height. if this is not taken care, you will get raster exception.

Finally to read the text from image, i use the tessaract.

//Read the text from image
String textInImage = extractText_Image(modifiedImageFilePath);
System.out.println("\n\nText in the image \""+modifiedImageFilePath+"\" is\n--------------------------");
System.out.println(textInImage);

/**
     * Function to extract text from image
     * @param filePath
     * @return
     */
 public  String extractText_Image(String filePath) {
		
    	File imageFile = new File(filePath);
		String TESS4J_FOLDER_PATH = "src"+getFileSeparator()+"main"+getFileSeparator()+"java"+getFileSeparator()+"tessdata";
		
		ITesseract instance = new Tesseract();
		instance.setDatapath(TESS4J_FOLDER_PATH);
		
		
		
		try {
			String result = instance.doOCR(imageFile);
			return result;
		} catch (TesseractException e) {
			System.err.println(e.getMessage());
			return "Error while reading image";
		}
	}
    
    public static String getFileSeparator() {
		return System.getProperty("file.separator");
	}


You need to download the tessdata for required language (English) and provide the path for the tessdata folder in your program.

This is the dependency i used to configure Tesseract jar files:

<dependencies>
   	<!-- https://mvnrepository.com/artifact/net.sourceforge.tess4j/tess4j -->
	<dependency>
	    <groupId>net.sourceforge.tess4j</groupId>
	    <artifactId>tess4j</artifactId>
	    <version>4.5.1</version>
	</dependency>
  </dependencies>

You may have to do multiple runs of the program to arrive at right values for A,B,C and D. One suggestion to help you in this regard: Capture the screenshot of the webpage once with your selenium test case. Create a separate Java project outside of your selenium workspace and copy the above program. Feed the screenshot to this java project and run it multiple iterations to arrive at right values for A,B,C and D.

To Summarize the idea:

  • Using takeScreenshot function provided by Selenium, capture and save the image in “originalImageFilePath“.
  • Then call the “cropImage” function to crop it and save the cropped image in different file in modifiedImageFilePath .
  • Finally, use the extractText_Image function to extract the text from modified image

You can find the entire project in my github repository.

If everything is working fine for you, you have reached the end of this article. If you are seeing some weird issues, read further.

Handling weird issues:

If you observe that the image is having reddish or brownish or greyish colored background after cropping, follow these 2 steps.

Step1: Modify the crop image function as below:

 private BufferedImage cropImage(BufferedImage src) {
        BufferedImage dest = src.getSubimage(500, 350,200, 350);
        Graphics2D drawer = dest.createGraphics() ;
        drawer.setBackground(Color.WHITE);
        drawer.clearRect(500, 350,200, 350);
        
        return dest; 
     }

Step2: Save the modified image as png instead of JPG. It doesn’t matter if your original file is in JPG format. But for this workaround to work, the modified image should be in png only.

ImageIO.write(crop_Image(original_Image), "png", modified_Image_File);

Selenium Tips: How to deal with iframes

If you are autoamting applications that make heave use of iframes, like the Salesforce Application, you need to definitely learn how to get the iframe details and use it in your selenium scripting such that it doesn’t turn out be a blocker in your progress.

So, like many Automation testers, if you use chrome browser to test your app then you need to follow the simple steps listed below to quickly get the iframe details and move forward with your scripting work.

Before even we start learning how to get iframe details, we need to be sure that the element we want our script to interact, is within the frame right? How to do that?

Well, that is pretty simple. All you will do is just Right click on that element. Most of the times, Chrome will show an option “View frame source” in the options list.

But don’t always rely on this quick tip. Sometimes, for reasons unknown to me, even if the element is within a iframe, chrome didn’t show these menu options. So, then how to double check? Simple, just click on the Inspect option. Once the Dev Tool window opens, click on console option.


See the small list box in pale pink color? if that list box says “top”, it means the element is not in a frame. But if it says anything other than “top” then, it means the element is within the frame.

So what the listbox shows? it might be the frame name or id. I’m not very sure at this time.

Now that we know how to make sure our element is within a iframe, lets move ahead to the actual topic of this post – how to get that iframe xpath? Very simple. Just follow these steps below:

Step1: Open the dev tools in chrome browser and click anywhere on the DOM. Now press Ctrl+F keys. You will see the search tool bar appear at the bottom of the dev tools window.

Step2: Now type //iframe. If there are iframes in your application, you will see the total number of iframes just beside the cancel button on the search bar.

Step3: Now, we are at a junction where we know that the element is within a iframe and we also know there is not just one but several iframes in the screen!! So, how to know which iframe our element is enclosed? see next step

Step4: Keep hitting the enter button. You will see various iframes getting highlighted with yellow color in DOM. At the same time, the corresponding frame in UI screen gets highlighted with pale blue color.

Step 5: So, what’s the final step? You got the iframe details from DOM. Just get the CSS or id or name properties from that and construct the xpath. Test your xpath in Console tab and you are all set.

Bonus tips:

Always make sure you switch to default content – driver.switchTo().defaultContent()

Once done with working on the element, don’t forget to switch back to default content.

JSON Testing with Java.. part 2

In my first post, I presented the code to get the flat map out of a JSON using Java Streams.

In this post, i will try to explain how the code works.

In the “JsonTesting” tester class, i use the jackson library to get the map created out of the String (which is internally of JSON structure, but Java doesn’t knows it, since i’ve not explicitly declared it as JSON object using any JSON library)

            ObjectMapper mapper = new ObjectMapper();
            TypeReference<Map<String, Object>> type = new TypeReference<Map<String, Object>>() {};
     
            //convert the json to ordinary map of string objects
            Map<String, Object> actualJsonMap = mapper.readValue(actualJSON, type);
            Map<String, Object> expectedJsonMap = mapper.readValue(expectedJSON, type);
             
         

if you take the below JSON example:

{
    "glossary": {
        "title": "example glossary",
		"GlossDiv": {
            "title": "S",
			"GlossList": {
                "GlossEntry": {
                    "ID": "SGML",
					"SortAs": "SGML",
					"GlossTerm": "Universal Generalized Markup Language",
					"Acronym": "SGML",
					"Abbrev": "ISO 8879:1986",
					"GlossDef": {
                        "para": "A meta-markup language, used to create markup languages such as DocBook.",
						"GlossSeeAlso": ["GML", "XML"]
                    },
					"GlossSee": "markup"
                }
            }
        }
    },
	"Summary": "end of JSON"
}

Then the output from line 50 (actualJsonMap) will be

{glossary={title=example glossary, GlossDiv={title=S, GlossList={GlossEntry={ID=SGML, SortAs=SGML, GlossTerm=Universal Generalized Markup Language, Acronym=SGML, Abbrev=ISO 8879:1986, GlossDef={para=A meta-markup language, used to create markup languages such as DocBook., GlossSeeAlso=[GML, XML]}, GlossSee=markup}}}}, Summary=end of JSON}

So the jackson just replaced the “:” (colon) symbol with = symbol and created an array of hashmaps

Coming to the flatten function in FlatMapUtil.Java

 public static Map<String, Object> flatten(Map<String, Object> map) {
    	
    	System.out.println(map.entrySet());
        return map.entrySet()
                .stream()
                .flatMap(FlatMapUtil::flatten)
                .collect(LinkedHashMap::new, (m, e) -> m.put("/" + e.getKey(), e.getValue()), LinkedHashMap::putAll);
    }

I have Sys out the map.entrySet(). It will print the following to console

[glossary={title=example glossary, GlossDiv={title=S, GlossList={GlossEntry={ID=SGML, SortAs=SGML, GlossTerm=Universal Generalized Markup Language, Acronym=SGML, Abbrev=ISO 8879:1986, GlossDef={para=A meta-markup language, used to create markup languages such as DocBook., GlossSeeAlso=[GML, XML]}, GlossSee=markup}}}}, Summary=end of JSON]

If you look at it closely, this collection has two hashmaps in it:

Hashmap1:

glossary={title=example glossary, GlossDiv={title=S, GlossList={GlossEntry={ID=SGML, SortAs=SGML, GlossTerm=Universal Generalized Markup Language, Acronym=SGML, Abbrev=ISO 8879:1986, GlossDef={para=A meta-markup language, used to create markup languages such as DocBook., GlossSeeAlso=[GML, XML]}, GlossSee=markup}}}}

HashMap2:

Summary=end of JSON

So the stream function in flatten method (1st one) will send each of these hashmaps one after the other iteratively.

The flatMap function needs one Lambda function, to return you a flatmap from the origninal map given to it. That’s what the line of code “.flatMap(FlatMapUtil::flatten)” is doing. it calls the next function – flatten, to work on this hashmaps one after the other

So, now lets see how the 2nd flatten function works, when it is called for the first time, with Hashmap1 as input., this is how the hashmap looks at the runtime

 private static Stream<Entry<String, Object>> flatten(Entry<String, Object> entry) {
    	
    	System.out.println("--------------\n");
    	StackTraceElement[] stacktrace = Thread.currentThread().getStackTrace();
    	StackTraceElement se = stacktrace[2];//maybe this number needs to be corrected
    	System.out.println("Calling function: "+se.getMethodName());
    	System.out.println("Flatten method entry Number: "+counter);
    	System.out.println("Entry is: "+entry);
    	counter+=1;
    	//System.out.println("--------------\n");
    	
        if (entry == null) {
            return Stream.empty();
        }

        if (entry.getValue() instanceof Map<?, ?>) {
        	System.out.println("Inside 1st if block. Entry.GetValue() is: "+ entry.getValue());
            Map<?, ?> properties = (Map<?, ?>) entry.getValue();
            System.out.println("Inside 1st if block. Properties Map is: "+ properties);
            return properties.entrySet()
                    .stream()
                    .flatMap(e -> flatten(new  java.util.AbstractMap.SimpleEntry<>(entry.getKey() + "/" + e.getKey(), e.getValue())));
        }

        if (entry.getValue() instanceof List<?>) {
        	System.out.println("Inside 2nd if block. Entry.GetValue() is: "+ entry.getValue());
            List<?> list = (List<?>) entry.getValue();
            return IntStream.range(0, list.size())
                    .mapToObj(i -> new  java.util.AbstractMap.SimpleEntry<String, Object>(entry.getKey() + "/" + i, list.get(i)))
                    .flatMap(FlatMapUtil::flatten);
        }

        return Stream.of(entry);
    }

When this (2nd) flatten function is called, control goes to 2nd if block inside it, since the boolean is evaluated to true

if (entry.getValue() instanceof Map<?, ?>) {
        	
            Map<?, ?> properties = (Map<?, ?>) entry.getValue();
            
            Stream<Entry<String, Object>> outputMap = properties.entrySet()
                    .stream()
                    .flatMap(e -> flatten2(new  java.util.AbstractMap.SimpleEntry<>(entry.getKey() + "/" + e.getKey(), e.getValue())));
          return outputMap;
        }
  1. entry is the Hashmap1
  2. properties is the value part of the entry map i.e {title=example glossary, GlossDiv={title=S, GlossList={GlossEntry={ID=SGML,……
  3. Notice that the value part in itself is a Hashmap. Lets call it as Sub Hashmap
  4. the lambda function inside the flatmap will be acting on the sub Hashmap Stream
  5. so entry.getKey() will give glossary
  6. After glossary, / is appended
  7. first hashmap of the SubHashmap is title=example glossary
  8. e.getKey() will give title
  9. e.getValue() will give example glossary
  10. so the whole expression will give glossary/title=example glossary
  11. Next, the return statement (inside of if block) is executed. So the control has to return to the calling function, which is the same lambda function inside 2nd If block, with glossary/title=example glossary as input
  12. The lambda function will in turn call the flatten(2nd) function again with glossary/title=example glossary as input
  13. This time when flatten function is executed with glossary/title=example glossary as input parameter to it, it will skip all if blocks, as it is not meeting any of the if block Boolean
    • Notice that the 2nd if block is having the Boolean that checks the value of the entry set. if that value is of type hashmap then only the Boolean evaluates to true.
  14. So it will reach the last return statement – return Stream.of(entry);
    • which evaluates to return Stream.of(glossary/title=example glossary);
  15. This resultant hashmap glossary/title=example glossary is now collected by the collector of flatten (1st one)
  16. The control has to go back to caller function, which is the lambda function in 2nd if block.
  17. And now, when the lambda function is called again, it will iterate to next element in e map – GlossDiv={title=S, GlossList={GlossEntry={ID=SGML,…..
  18. So this time the Lambda function will call the flatten(2nd one) again with the hashmap as glossary/GlossDiv={title=S, GlossList={Glo….
  19. This will repeat until the Streamer in the labmbda function inside the 2nd if block is done going over every single Hashmap entry of Sub Hashmap.
  20. Once done, the streamer inside first flatten will iterate to the HashMap 2 and the process i completed.

Reading Text from Image using Java

In this post, we will see how to extract text from an image using OCR libraries in Java

In the program below, i’ve used Tesseract with Tess4j library. This is how the pom looks

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
   	<!-- https://mvnrepository.com/artifact/net.sourceforge.tess4j/tess4j -->
	<dependency>
	    <groupId>net.sourceforge.tess4j</groupId>
	    <artifactId>tess4j</artifactId>
	    <version>4.5.1</version>
	</dependency>
  </dependencies>

Once you configure the POM, you can use this below program to extract the text out of image file.

Few important points to note:

  • There is no perfect OCR library as of today. So expect to see some junk chars or issues in the output
  • Some posts say that you have to set environment variable to “tessdata” repository. This is no longer required with version 4 and above. Instead, you will set the pointer to the repo inside the code itself. See the program.
  • You cannot get output if the text is skewed (not straight up). To overcome this problem, you will have to get the image processed to make it straight and extract the text
  • For better results, make sure that image is big enough. small images will not yeild good results. The recommendation is to have image with 300 DPI.
  • You can do some image processing, before you extract the text. This will improve the output. For ex. Grayscaling, increasing image dimension, removing the skew etc. Look up in google to get the code for performing these. You can look for libraries like OpenCV, to do these processing ops on image.
import java.io.File;

import net.sourceforge.tess4j.ITesseract;
import net.sourceforge.tess4j.Tesseract;
import net.sourceforge.tess4j.TesseractException;

/**
 * Class file to extract text from Image
 * @author i366155
 *
 */
public class ImageTesting 
{
    public static void main( String[] args ){
    	
        ImageTesting it = new ImageTesting();
        it.testImage();
        
    }
    
    
    public void testImage() {
    	String imageFilePath = "src"+getFileSeparator()+"main"+getFileSeparator()+"java"+getFileSeparator()+"files"+getFileSeparator()+"image5.jpg";
    	
    	String textInImage = extractImage(imageFilePath);
    	
    	System.out.println("\n\nText in the image \""+imageFilePath+"\" is\n--------------------------");
    	System.out.println(textInImage);
		
    	
    }
    
    public  String extractImage(String filePath) {
		
    	File imageFile = new File(filePath);
		String TESS4J_FOLDER_PATH = "src"+getFileSeparator()+"main"+getFileSeparator()+"java"+getFileSeparator()+"tessdata";
		
		ITesseract instance = new Tesseract();
		instance.setDatapath(TESS4J_FOLDER_PATH);
		
		
		
		try {
			String result = instance.doOCR(imageFile);
			return result;
		} catch (TesseractException e) {
			System.err.println(e.getMessage());
			return "Error while reading image";
		}
	}
    
    public static String getFileSeparator() {
		return System.getProperty("file.separator");
	}
}

JSON Testing with Java

When we want to test a JSON in Java, the first thing that comes to mind is deserialization on JSON into POJO. This is not just complex, but time consuming too.

here we will see how to test a JSON without deserializing it or parsing it!!

I’ve used only 2 libraries here – Jackson library and Guava library. This is the Pom.xml of my project.

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
    <dependency>
	    <groupId>junit</groupId>
	    <artifactId>junit</artifactId>
	    <version>3.8.1</version>
	    <scope>test</scope>
    </dependency>
    
    <dependency>
	    <groupId>com.fasterxml.jackson.core</groupId>
	    <artifactId>jackson-databind</artifactId>
	    <version>2.9.8</version>
	</dependency>
	
	<!-- https://mvnrepository.com/artifact/com.google.guava/guava -->
	<dependency>
	    <groupId>com.google.guava</groupId>
	    <artifactId>guava</artifactId>
	    <version>19.0</version>
	</dependency>
	
  </dependencies>
</project>

I referred this beautiful blog post to come up with this solution.

This solution works for Java 8 and above as we are going to use the Lambda expressions and Java streams, introduced from Java 8. if you have configured latest version of Java JDK in your build path then, right click on your project >properties. Search for JRE and click on Java Compiler. Change the Compiler compliance level to 1.8.

The first step is to convert the JSON into a Flat map stream. To do that, simply add this class file to your package folder in project.

import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.stream.IntStream;
import java.util.stream.Stream;


/**
 * Library to convert a complex JSON into flat map
 * @author i366155
 *
 */
public class FlatMapUtil {

	

    private FlatMapUtil() {
        throw new AssertionError("No instances for you!");
    }

    public static Map<String, Object> flatten(Map<String, Object> map) {
        return map.entrySet()
                .stream()
                .flatMap(FlatMapUtil::flatten)
                .collect(LinkedHashMap::new, (m, e) -> m.put("/" + e.getKey(), e.getValue()), LinkedHashMap::putAll);
    }

    private static Stream<Entry<String, Object>> flatten(Entry<String, Object> entry) {

        if (entry == null) {
            return Stream.empty();
        }

        if (entry.getValue() instanceof Map<?, ?>) {
            Map<?, ?> properties = (Map<?, ?>) entry.getValue();
            return properties.entrySet()
                    .stream()
                    .flatMap(e -> flatten(new  java.util.AbstractMap.SimpleEntry<>(entry.getKey() + "/" + e.getKey(), e.getValue())));
        }

        if (entry.getValue() instanceof List<?>) {
            List<?> list = (List<?>) entry.getValue();
            return IntStream.range(0, list.size())
                    .mapToObj(i -> new  java.util.AbstractMap.SimpleEntry<String, Object>(entry.getKey() + "/" + i, list.get(i)))
                    .flatMap(FlatMapUtil::flatten);
        }

        return Stream.of(entry);
    }
}

Next, we will see how to use this stream of Map for our testing. Let’s see different scenarios of testing.

Scenario 1

Here, we will compare 2 JSON objects :

  • Expected JSON – Source of truth
  • Actual JSON – JSON to be tested against Expected

The below program shows how to compare the two JSON and

  • Print the entries that are present only in Actual JSON
  • Print the entries that are present only in Expected JSON
  • Entries that mismatch – i.e. values of the keys are not matching – it is case sensitive
  • Entries that are common between the two JSON

To do this comparison, we will use Maps.difference function of Guava library.

import java.io.File;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Scanner;

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.collect.MapDifference;
import com.google.common.collect.Maps;



public class JsonTesting
{
	public static void main( String[] args ){
		
		JsonTesting ju = new JsonTesting();
		ju.compareJsonPrintDiff();
		
	}

	
	
	/**
	 * @function to compare 2 json - option2
	 */
	public void compareJsonPrintDiff() {
		
		/*
		 * Compares two json and prints the difference
		 * prints entries only on left, only on right and common entries
		 * prints the difference - key mismatch, value mismatch
		 */
		try {
			
			//get the path of json files
			String actualJSONfilePath = "src"+getFileSeparator()+"main"+getFileSeparator()+"java"+getFileSeparator()+"files"+getFileSeparator()+"Actual.json";
			String expectedJSONfilePath = "src"+getFileSeparator()+"main"+getFileSeparator()+"java"+getFileSeparator()+"files"+getFileSeparator()+"Expected.json"; 
	
			//read the json as string
			String actualJSON = getJSONfromFile(actualJSONfilePath);
			String expectedJSON = getJSONfromFile(expectedJSONfilePath);
			
			//Comparison operation starts
			ObjectMapper mapper = new ObjectMapper();
			TypeReference<Map<String, Object>> type = new TypeReference<Map<String, Object>>() {};
	
			//convert the json to ordinary map of string objects
			Map<String, Object> actualJsonMap = mapper.readValue(actualJSON, type);
			Map<String, Object> expectedJsonMap = mapper.readValue(expectedJSON, type);
			
			//convert the string object map to flat map (containing the entire address as key and its value"
			Map<String, Object> actualJsonFlatMap = FlatMapUtil.flatten(actualJsonMap);
			Map<String, Object> expectedJsonFlatMap = FlatMapUtil.flatten(expectedJsonMap);
	
			/*
			 * Compute the difference between the 2 flat maps using Guava library
			 * left map is actual Json to be tested
			 * right map is Expected Json - source of truth
			 */
			MapDifference<String, Object> difference = Maps.difference(actualJsonFlatMap, expectedJsonFlatMap);
			
			//Printing of computation results
			System.out.println("Entries only on left\n--------------------------");
			difference.entriesOnlyOnLeft().forEach((key, value) -> System.out.println(key + ": " + value));
	
			System.out.println("\n\nEntries only on right\n--------------------------");
			difference.entriesOnlyOnRight().forEach((key, value) -> System.out.println(key + ": " + value));
	
			System.out.println("\n\nEntries differing\n--------------------------");
			difference.entriesDiffering().forEach((key, value) -> System.out.println(key + ": " + value));
	
			System.out.println("\n\nEntries in common\n--------------------------");
			difference.entriesInCommon().forEach((key, value) -> System.out.println(key + ": " + value));
			
		}catch(Exception e) {
			e.printStackTrace();
		}
	}
	
	/**
	 * @Function to get file separator (instead of using \ )
	 */
		
	public static String getFileSeparator() {
		return System.getProperty("file.separator");
	}

	/**
	 * @function to read data from file
	 * @param filePath
	 * @return
	 */
	public String getJSONfromFile(String filePath) {
		
		//Uses scanner library to read entire file, without using loop

		String fileContent=null;

		try {
			File file = new File(filePath); 
			Scanner sc = new Scanner(file); 

			// we just need to use \\Z as delimiter 
			sc.useDelimiter("\\Z"); 
			fileContent = sc.next();

			sc.close();

		}catch(Exception e) {
			fileContent = "error. Check stacktrace";
			e.printStackTrace();
		}

		return fileContent; 

	}


}

Expected.json file content

{
    "glossary": {
        "title": "example glossary",
		"GlossDiv": {
            "title": "S",
			"GlossList": {
                "GlossEntry": {
                    "ID": "SGML",
					"SortAs": "SGML",
					"GlossTerm": "Standard Generalized Markup Language",
					"Acronym": "SGML",
					"Abbrev": "ISO 8879:1986",
					"GlossDef": {
                        "para": "A meta-markup language, used to create markup languages such as DocBook.",
						"GlossSeeAlso": ["GML", "XML"]
                    },
					"GlossSee": "markup"
                }
            }
        }
    }
}

Actual.Json content. Note that the “GlossTerm” key’s value is different and also there is extra entry “Summary” in this JSON.

{
    "glossary": {
        "title": "example glossary",
		"GlossDiv": {
            "title": "S",
			"GlossList": {
                "GlossEntry": {
                    "ID": "SGML",
					"SortAs": "SGML",
					"GlossTerm": "Universal Generalized Markup Language",
					"Acronym": "SGML",
					"Abbrev": "ISO 8879:1986",
					"GlossDef": {
                        "para": "A meta-markup language, used to create markup languages such as DocBook.",
						"GlossSeeAlso": ["GML", "XML"]
                    },
					"GlossSee": "markup"
                }
            }
        }
    },
	"Summary": "end of JSON"
}

When you run the program, the output will be

Entries only on left
--------------------------
/Summary: end of JSON


Entries only on right
--------------------------


Entries differing
--------------------------
/glossary/GlossDiv/GlossList/GlossEntry/GlossTerm: (Universal Generalized Markup Language, Standard Generalized Markup Language)


Entries in common
--------------------------
/glossary/title: example glossary
/glossary/GlossDiv/title: S
/glossary/GlossDiv/GlossList/GlossEntry/ID: SGML
/glossary/GlossDiv/GlossList/GlossEntry/SortAs: SGML
/glossary/GlossDiv/GlossList/GlossEntry/Acronym: SGML
/glossary/GlossDiv/GlossList/GlossEntry/Abbrev: ISO 8879:1986
/glossary/GlossDiv/GlossList/GlossEntry/GlossDef/para: A meta-markup language, used to create markup languages such as DocBook.
/glossary/GlossDiv/GlossList/GlossEntry/GlossDef/GlossSeeAlso/0: GML
/glossary/GlossDiv/GlossList/GlossEntry/GlossDef/GlossSeeAlso/1: XML
/glossary/GlossDiv/GlossList/GlossEntry/GlossSee: markup

SCENARIO 2

Here we don’t have source of truth JSON. We just want to test the values in each of the keys of the actual JSON.

Below is the program written to do this test

import java.io.File;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Scanner;

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.collect.MapDifference;
import com.google.common.collect.Maps;



public class JsonTesting
{
	public static void main( String[] args ){
		
		JsonTesting ju = new JsonTesting();
		ju.convertJsonToMapAndValidate();

	}

	
	
	
	
	/**
	 * @Function to validate JSON
	 */
	
	public void convertJsonToMapAndValidate() {
		
		/*
		 * converts the JSON into flat map (key-value pair)
		 * we can iterate through each key-value pair and validate
		 */
		try {
			
			//get the path of json files
			String actualJSONfilePath = "src"+getFileSeparator()+"main"+getFileSeparator()+"java"+getFileSeparator()+"files"+getFileSeparator()+"Actual.json";
			
			//read the json as string
			String actualJSON = getJSONfromFile(actualJSONfilePath);
			
			
			ObjectMapper mapper = new ObjectMapper();
			TypeReference<Map<String, Object>> type = new TypeReference<Map<String, Object>>() {};
	
			//convert the json to ordinary map of string objects
			Map<String, Object> actualJsonMap = mapper.readValue(actualJSON, type);
			
			
			//convert the string object map to flat map (containing the entire address as key and its value"
			Map<String, Object> actualJsonFlatMap = FlatMapUtil.flatten(actualJsonMap);
			
	
			//iterate through flat map and validate
			System.out.println("\n\nValidation\n--------------------------");
			for (Entry<String, Object> entry : actualJsonFlatMap.entrySet()) {
	            //System.out.println("Key = " + entry.getKey() + ", Value = " + entry.getValue());
	            if(entry.getKey().equals("/glossary/GlossDiv/GlossList/GlossEntry/GlossTerm")) {
	            	if(entry.getValue().equals("Universal Generalized Markup Language")) {
	            		System.out.println("value in key - /glossary/GlossDiv/GlossList/GlossEntry/GlossTerm is matching");
	            	}
	            }
			}
		
		}catch(Exception e) {
			e.printStackTrace();
		}
	}
	

	public static String getFileSeparator() {
		return System.getProperty("file.separator");
	}

	/**
	 * @function to read data from file
	 * @param filePath
	 * @return
	 */
	public String getJSONfromFile(String filePath) {
		
		//Uses scanner library to read entire file, without using loop

		String fileContent=null;

		try {
			File file = new File(filePath); 
			Scanner sc = new Scanner(file); 

			// we just need to use \\Z as delimiter 
			sc.useDelimiter("\\Z"); 
			fileContent = sc.next();

			sc.close();

		}catch(Exception e) {
			fileContent = "error. Check stacktrace";
			e.printStackTrace();
		}

		return fileContent; 

	}


}

The output will be

Validation
--------------------------

value in key - /glossary/GlossDiv/GlossList/GlossEntry/GlossTerm is matching

For detailed explanation on how the code logic works, you can refer to this post.

JSON parsing

There are lot of options to parse a JSON string in JAVA. Most of them ask you to deserialize the JSON object (partially or completely), either before hand or during the runtime (GSON).

But for light weight use, you may not be interested to do the complex work of deserialization. In that case, one of the best option is JsonPath library. Google JsonPath for latest version of the library and dependency.

Below i’ve give a sample JSON and the program to parse it and pick the value from desired keys.

{
    "pageInfo": {
            "pageName": "Homepage",
            "logo": "https://www.example.com/logo.jpg"
    },
    "posts": [
            {
                "post_id": "0123456789",
                "actor_id": "1001",
                "author_name": "Jane Doe",
                "post_title": "How to parse JSON in Java",
                "comments": [],
                "time_of_post": "1234567890"
            }
    ]
}
import com.jayway.jsonpath.JsonPath;

public class JsonParser {
	
	public static void main(String args[]) {
		
		
		JsonParser jp = new JsonParser();
		String json = jp.jsonTemplate();
		
		System.out.println("Json is:");
		System.out.println(json);
		
		String pageName = JsonPath.read(json, "$.pageInfo.pageName");
        System.out.println(pageName);

        Integer posts = JsonPath.read(json, "$.posts.length()");

        for(int i=0; i < posts; i++) {
            String post_id = JsonPath.read(json, "$.posts[" + i + "].post_id");
            System.out.println(post_id);
        }
		
		
	}
	
	public String jsonTemplate() {
		
		String template ="{\r\n" + 
				"    \"pageInfo\": {\r\n" + 
				"            \"pageName\": \"Homepage\",\r\n" + 
				"            \"logo\": \"https://www.example.com/logo.jpg\"\r\n" + 
				"    },\r\n" + 
				"    \"posts\": [\r\n" + 
				"            {\r\n" + 
				"                \"post_id\": \"0123456789\",\r\n" + 
				"                \"actor_id\": \"1001\",\r\n" + 
				"                \"author_name\": \"Jane Doe\",\r\n" + 
				"                \"post_title\": \"How to parse JSON in Java\",\r\n" + 
				"                \"comments\": [],\r\n" + 
				"                \"time_of_post\": \"1234567890\"\r\n" + 
				"            }\r\n" + 
				"    ]\r\n" + 
				"}";
		
		return template;
	}

}

Java – make a window active

In your automation journey, if you come across a situation where you want to know if the process (exe) is running or not, if that process window (like putty window, excel window, notepad window etc…) is active or in the background, and bring the window to foreground or maximize it, read through this post.

JNA library provides this ability for us. you need to extend the interface of the JNA StdCallLibrary and then use the functions available to do the job

Note: The below code works with Selenium Sever standalone jar. The same code didnt work fine when i used the JNA 5.x jar file! i was getting some runtime error (which was already discussed in Stackoverflow thread and there was no solution for it). So better use selenium server standalone jar file.

import com.sun.jna.Native;
import com.sun.jna.platform.win32.WinDef.HWND;
import com.sun.jna.win32.StdCallLibrary;

Public class windowsTesting {	

public  interface User32 extends StdCallLibrary {
	    final User32 INSTANCE = (User32) Native.loadLibrary ("user32", User32.class);
	    HWND FindWindowA(String winClass, String title); 
	    boolean ShowWindow(HWND hWnd, int nCmdShow);
	    boolean SetForegroundWindow(HWND hWnd);

	}

	
	public  void validateActiveWindow(){
		
		final User32 user32 = User32.INSTANCE;
			
		HWND hwnd  = user32.FindWindowA
		       (null, "Notepad"); // window title
	
		if (hwnd == null) {
			//if putty is not running, terminate the TC execution
			System.out.println("Notepad is not running");
			
			System.exit(0);
		}
		else{
			//if putty is running, make it active window
			//System.out.println("Notepad is  running");
			
			User32.INSTANCE.ShowWindow(hwnd, 9 );        // SW_RESTORE
			User32.INSTANCE.SetForegroundWindow(hwnd);   // bring to front
		}
		
		//set to null, to help GC cleanup the object
		hwnd = null;
	}
}
	

Selenium Grid – Issue resolution

In this post we will see the root cause of the below issue and the resolution:

org.openqa.grid.common.exception.GridException: Session [null] not available and is not among the last 1000 terminated sessions.

When i searched various forums about this issue, i got a hint that this happens when the selenium standalone server jar file version is not same between the source and destination.

That means: Make sure you have same version of the jar file in

  1. Hub
  2. Nodes
  3. your local machine from where you trigger the execution

The 3rd one is tricky! I was under impression that the business of execution is between Hub and Node and it doesn’t matter what version of server jar file i’ve in my local machines. But i was proved wrong. It does matter!!

So when i synced up the jar file in all 3 places, it fixed the issue.