Sunday, 12 January 2020

JMeter - Selenium


Objective
POC to perform test with JMeter Selenium Web driver that helps in analysis of client side performance

Observation
We have observed following four major hurdles during the process
1.       JMeter WebDriver Compatibility
2.       Code Reusability
3.       Sub Sample Timer
4.       Response time variation

JMeter WebDriver Compatibility with Application
JMeter Webdriver sampler has limited set of options during instantiation. Application automation will be blocked with this limitation. One such example is popup to accept/reject notifications during login.
This pop should be disabled before performing any actions in the web page and there is no such option to handle in JMeter by default.
Resolution
  • We have overcome this by tweaking code in the plugin and adding required chrome options that were picked during instantiation.


Code Maintenance 
With script editor provided by Jmeter Webdriver, there are following 3 challenges
1.       Duplicate locators for same web elements
2.       Test script maintenance
3.       readability 
Resolution
  • We have implemented Page Object Model for webpage action automation. This has been achieved by importing the web page objects to the JMeter.


Sub Sample Timer 
JMeter provides a function to capture timing for individual parts of the script. But, this data will only be available in View Results tree element and not in Results table or Result file. 
For example for a transaction Login with LoginPage, TenantSelectionPage, HomePage as sub areas, following is the result
Output - View Results tree  
                                Machine generated alternative text: LoginPage ogn age TenantSeIectonPage Homepage FileCabinetPage 2. NavigateToFiIeCabinet DynamicReportFoIderPage 3. NavigateToFoIder DynamicReportPage 4RUnDynamicReport aackToLoginPage 5. Logout Sampler result Request Resguynse data Thread Name: Thread Group I-I sample start: 2019-10-22 Load time: 27689 Latency: O Size in bytes: O Headers size in bytes: O Body size in bytes: O Sample Count: I Error Count: O Response code: Resguynse message: Resw•se headers: SampleResuI t fields: ContentType: Da taEncoding : null

Output - View Results Table 
                                Machine generated alternative text: Status LoginPage I. Login FileCabinetPage 2. NavigateToFiIeCabinet DynamicReportFoIderPage 3. NavigateToFoIder DynamicReportPage 4 RunDynamicReport aa ckToLoginP ag e 5. Logout Sample Time (ms) 30993 30993  
Resolution
  • We can distribute functionality to individual samplers and grouped them under transaction controller. Below is the result:
Machine generated alternative text: LoginPage enantSeIectonPage Homepage I. Login Sample Tme (ms) 10332 Status


Difference in response time 
We have observed variance in response times due to the difference in the request simulation by JMeter with and without WebDriver Sampler. Following table represents the response time in secs for 5 transactions. We can observe response time is high in case of Jmeter when compared to Selenium Webdriver.

Transaction Response Time (sec)
               
Transaction
JMeter Selenium Webdriver
Jmeter
Count
Avg
Min
Max
Stdev
Count
Avg
Min
Max
Stdev
1.Login
10
6.5
5.51
10.41
1.4
10
1.89
1.46
2.21
0.25
2.NavigateToFileCabinet
10
1.35
1.17
1.56
0.11
10
0.62
0.49
0.72
0.09
3.NavigateToFolder
10
0.39
0.35
0.41
0.02
10
0.08
0.07
0.1
0.01
4.OpenDashboard
10
6.23
5.88
7.85
0.6
10
9.96
3.96
12.48
3.08
4.RunReport
10
3.2
2.92
4.15
0.45
10
3.88
3.35
4.53
0.46
5.Logout
10
0.82
0.68
1.03
0.1
10
0.22
0.16
0.26
0.03


This is due to following reasons:
a.       Way of handling Async requests by client
b.       Number of concurrent web server connections created by client
    1. Following table represents comparison of connection creations to WebServer from client for Login transaction.

Chrome
JMeter

c.       Handling the request queue by client. We can’t simulate the request queue of the browser from JMeter.

Resolution
    1. Pick max of response times for Async
Following is the response time comparison after picking max values
Transaction
JMeter Selenium Webdriver
Jmeter
Count
Avg
Min
Max
Stdev
Count
Avg
Min
Max
Stdev
1.Login
10
6.5
5.51
10.41
1.4
10
1.89
1.46
2.21
0.25
2.NavigateToFileCabinet
10
1.35
1.17
1.56
0.11
10
0.62
0.49
0.72
0.09
3.NavigateToFolder
10
0.39
0.35
0.41
0.02
10
0.08
0.07
0.1
0.01
4.OpenJarvisDashboard
10
6.23
5.88
7.85
0.6
10
4.14
2.47
5.24
0.88
4.RunDynamicReport
10
3.2
2.92
4.15
0.45
10
3.88
3.35
4.53
0.46
5.Logout
10
0.82
0.68
1.03
0.1
10
0.22
0.16
0.26
0.03
    1. We couldn’t find resolution for simulation  of browser request queue and browser concurrent connections.

at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)