Silk Performer 2008 Training Courseware By Softsmith Infotech www.softsmithinfotech.

com

Silk Performer Courseware

1 of 33

Softsmith Infotech

Table of Contents TABLE OF CONTENTS .........................................................................................2 INTRODUCTION ....................................................................................................3

Need For Load Testing..................................................................................................................................... 3 Silk Performer Features................................................................................................................................... 3 Load Test Process Steps ................................................................................................................................... 3

LOAD TEST PLANNING........................................................................................4

Functional Testing Vs Load Testing............................................................................................................... 4 Load Testing Checklist..................................................................................................................................... 4 Load Test Guidelines........................................................................................................................................ 5

PREPARING THE WORK BENCH ........................................................................5 PROFILES ..............................................................................................................8 CREATION OF SCRIPTS ......................................................................................9 PARAMETERIZATION OF SCRIPTS..................................................................12 WORKLOADS ......................................................................................................23 ANALYSIS............................................................................................................28 ACTUAL BASE LINE REPORT...........................................................................32

Silk Performer Courseware

2 of 33

Softsmith Infotech

Introduction
Need For Load Testing
Any multi-user application needs to face the concurrent access some day or the other. Before deploying the application and then exposing the application for multiple users it is better we test it and then do the deployment. This process is load testing. Minimal Infrastructure - We cannot gather hundreds or thousands of people to carry out concurrent user tests and this will not be possible for large number of users for longer time Reliable - Tests perform precisely the same operations each time they are run, thereby eliminating human error. Repeatable - We can test how the application reacts after repeated execution of the same operations, for longer durations for many days Programmable - We can program sophisticated tests that bring out hidden information. Comprehensive - We can build a suite of tests that covers every feature in our application. Reusable - We can reuse tests on different versions of an application, even if the user interface changes.

Silk Performer Features

The following are the key features of SP (Silk Performer). Use Add-ins to support multiple environments. These add-ins are the ones that enable SP to recognize different protocols. Record vuser scripts and debug scripts (Scripts under Projects) Configure and Run scenarios (Work Load under Projects) Schedule and run scripts (Work Load under Projects) Run tests in a distributed manner (Agents under Projects) Analyze graphs (Explore Time Series, Monitor Server under Results)

Load Test Process Steps

Plan Create scripts Create scenarios Run & monitor scenarios Analyze results

Silk Performer Courseware

3 of 33

Softsmith Infotech

Load Test Planning

Identify most frequently used transactions Identify potential number of users Identify potential number of concurrent users Apply 10:1 or 5:1 ratio for logged-in Vs concurrent users Identify the production platform size and configuration Identify the data to be used for testing Identify the different real-time usage combinations of test scenarios Identify the load test run duration Identify what kind of information is transmitted between server and client Plan load testing only after functional stability of the product is achieved Discuss with other stakeholders like network admin, database admin, server admin and others on what information is required for them Chalk out the software configurations/settings for web server, app server and database server

Functional Testing Vs Load Testing

If preconditions are met and steps are followed, function test results are defined. Load test results are always unpredictable Functional test results do not change more than 5% when moved from one configuration to the other. Load Test results may even nose-dive! Functional test happens on a daily basis; but load test is not that frequent Load test results depend on database volume as well and they change when number of users change

Load Testing Checklist

Do we have the near-production hardware configuration? If not what is the delta between test hardware and production hardware? Is the tool capable of recording the requests based on the protocols used by the application (e.g. HTTPS) and able to replay the same? Is the product functionally cleared before load testing? Can we get numbers on the user counts from customer, based on past records? Is the data pool containing unique data? Is the trace log enabled for database and web servers? Are the requests distributed equally to different boxes? Is there a load balancer? Is there a facility in the tool to mimic different line speeds? Is there a facility in the tool to mimic different browser versions? Is there a facility in the tool to selectively log messages? Is there a facility in the tool to export the data in xls format? Is there a facility in the tool to auto-synchronize concurrent requests? If the application uses queues, the queue size must be monitored during test runs. Do the tests need runs with and without proxy servers? Do the tests need runs with and without firewalls?

Silk Performer Courseware

4 of 33

Softsmith Infotech

Load Test Guidelines

Number of users Vs response time must not be linear Stress test needs to be done for shorter durations and not for longer durations To the extent possible, let the data pool contain more unique data than what is needed The load generating client machines must not be operated at capacities beyond 80% for CPU and memory Avoid enabling detailed log information in the tool which will take more disk IO in the client machines Script must be parameterized for accessing the same application with different configurable URLs. So if the application is moved form one box to the other, the script can be reused Wherever needed, use rendezvous points to synchronize the requests before any form submission actions in the script. This ensures the simultaneous hits at the time of form submission If there is a possibility, disable downloading image files as image files are not downloaded every time in real time usage. Check the consistency of response time over a period of elapsed time and compare it with different test runs All successful requests must have been submitted and the log files must match. If the requests trigger data base operations, the same must have been recorded in database. The queues size must be minimal at any given point of time. Most of the time the database and the business logic layer need to be doubted first before the web server is doubted. Refer to Microsoft web URLs: http://msdn.microsoft.com/library/default.asp?url=/library/enus/dnpag/html/SCAG-CH08.asp, for finer details.

Preparing the Work Bench

Go to SilkPerformer Workbench, create a project in the silk performer Each project can have multiple scripts, each script being one business scenario. Each Script can have multiple transaction, Base transactions are Tinit, TMain, TShutdown, In between Tinit and TShutdown, We can have as many transactions as possible.

Silk Performer Courseware

5 of 33

Softsmith Infotech

Application type needs to be selected based web or Sap or Citrix etc. Silk performer supports following

Each type is nothing but different protocols which can be recognized by the performance tool. Each project should be associated to one of these protocols. Each project is associated with 1. 2. 3. 4. 5. 6. Profiles Scripts Include files Data files Agents Work loads

After the creation of projects

Silk Performer Courseware

6 of 33

Softsmith Infotech

System setting should be set as below, This setting are under Settings-System from menu

Silk Performer Courseware

7 of 33

Softsmith Infotech

Profiles
Profiles are nothing but the record and Replay setting, This can be configured. Profile has following setting for both Record and Replay 1. 2. 3. 4. 5. 6. Scripts Simulation Results Internet Web Terminal Client etc.

Scripts has General, Protocol, Java

Silk Performer Courseware

8 of 33

Softsmith Infotech

Simulation has Simulation (Run time Settings) and Errors

Results has 1. Time Series 2. Monitoring 3. True Log 4. Logging 5. Internet Logging 6. ARM 7. Hook Logging The others are based on the protocol we use for the application to be load tested.

Creation of Scripts
Scripts is generally created using the record command (Model Record)

Silk Performer Courseware

9 of 33

Softsmith Infotech

After recording the generated script look like below

Silk Performer Courseware

10 of 33

Softsmith Infotech

Model Script Records the script to the silk performer Try Script To ensure the recorded script is working properly.

In Try Script, The entire script will run as single user and it ensures the correctness of the script to parameter data, Dynamic data used by the script, proper proxy setting.

Silk Performer Courseware

11 of 33

Softsmith Infotech

Silk performer shows the run script in the true log window.

Parameterization of Scripts
To parameterize the data, select the data that needs to be parameterized

Right click and select the customize value

Silk Performer Courseware

12 of 33

Softsmith Infotech

Silk Performer Courseware

13 of 33

Softsmith Infotech

After finishing the parameterization, the script changes with following commands

Silk Performer Courseware

14 of 33

Softsmith Infotech

After the begin in transaction TMain, FileGetRndRow(hDataFile1); got inserted and below under FORM001 Variable q got commented and inserted new value for variable q

In the DataFiles, CSV file got inserted after the creation of parameter to the script We can use the same file for other parameter data.

Random run took the Qabed this time of try script. This ensures the right parameterization.

Creation of Scripts with Manual Correlation Following are the commands that needs to be used for manual correlation, Manual correlations are used to get the list box items, Hidden Session id etc. WebParseDataBoundEx(out sResult : string,

in nMaxResultLen : number optional, in sLeftBoundary : string optional, in nLeftOccurrence : number optional,

Silk Performer Courseware

15 of 33

Softsmith Infotech

in sRightBoundary : string optional, in nOptions in nDocNum : number optional, : number optional,

out nBytesParsed : number optional);

Return value

none

Parameter
sResult

Description

String variable that receives the string between the specified boundary strings. nMaxResultLen Maximum length of the string to return (optional). If this parameter is omitted or set to STRING_COMPLETE all available data is stored in sResult. sLeftBoundary Left boundary of the HTML content to compare. nLeftOccurrence The sLeftBoundary has to be found nLeftOccurrence times, before the copy process starts and the right boundary is searched (optional). The default value is one. Provide WEB_OCCURENCE_LAST to specify the last occurrence. sRightBoundary Right boundary of the HTML content to compare. nOptions (optional)

WEB_FLAG_CASE_SENSITIVE. If this flag is set the string compare operation is case sensitive. WEB_FLAG_IGNORE_WHITE_SPACE. If this flag is set all white spaces are ignored . WEB_FLAG_DONT_FORCE_LOAD. Specify this option to enable caching for subsequent request. Note that nothing is parsed if the specified table is not loaded (cache hit). WEB_FLAG_SYNCHRON. Parsing operations with this flag set, are done in a sequential way. The second parsing operation is not executed before the first has been completed. WEB_FLAG_ALL_RESPONSES. If this flag is specified all server responses are scanned (even redirection responses, which are normally not displayed by a browser). WEB_FLAG_INCLUDE_EMBEDDED. If this flag is specified, even embedded documents can be specified by the nDocNum parameter. Normally the number of a document is defined by the occurrence of the source definition in an HTML document (src=...). If this flag is specified, every embedded object increases this counter, which assigns higher numbers to subsequent frames. WEB_FLAG_INCLUDE_HEADER. If this flag is specified the response header can also be verified. WEB_FLAG_HEADER_ONLY. If this flag is specified only

Silk Performer Courseware

16 of 33

Softsmith Infotech

the response header is verified. WEB_FLAG_RULE. Specify this flag to perform the parsing operation in every subsequent web function. After every web function the specified output variables are set to the new value. If you want to use the variables in other transactions or in event handler functions use global variables! Call WebCancelAllRules() do disable all verification and parsing rules.

nDocNum

nBytesParsed

Note: The option WEB_FLAG_RULE should only be used in the INIT transaction or in combination with the WebCancelAllRules function! Specifies the document to parse (optional). Specify WEB_DOC_ALL if you want to parse all documents. If this parameter is omitted, the first document gets parsed. (see above definition FLAG_INCLUDE_EMBEDDED) Variable receiving the number of bytes actually parsed (optional).

Example for Single value rRndUniN1 this is the random value with in a range defined as, this is used to take the occurrence between Left boundary Value= and Right boundary \ dclrand rRndUniN1 : RndUniN (27..30);

WebParseDataBoundEx(sParseDataVar1, STRING_COMPLETE, ToEncoding("value=\""), rRndUniN1, ToEncoding("\""), WEB_FLAG_IGNORE_WHITE_SPACE | WEB_FLAG_CASE_SENSITIVE, 1);

WebPageUrl("http://192.168.1.107/SPCG/AddActionItem.aspx", "Action Item (#1)", SPCG_ADDACTIONITEM_ASPX012); Print("sParseDataVar1: " + FromEncoding(sParseDataVar1)); Writeln("sParseDataVar1: " + FromEncoding(sParseDataVar1)); Syntex for Multiple values in an array
Include file

WebAPI.bdh
Syntax

WebParseDataBoundArray( inout saResult in nMaxCount : number,

: array of string,

in sLeftBoundary : string, in sRightBoundary : string allownull, out nFound : number optional,

Silk Performer Courseware

17 of 33

Softsmith Infotech

in nOptions in nSkip in nDocNum in nMaxLen

: number optional, : number optional, : number optional, : number optional);

Parameter
saResult

Description

Array of string variable that receives all the strings between the specified boundary strings. nMaxCount Maximum number of strings copied into the provided array. This value must be less than or equal to the size of the array. sLeftBoundary Left boundary to compare. sRightBoundary Right boundary. After the sLeftBoundary has been found, all data is copied into the actual string parameter until the right boundary is found (optional). If this parameter is omitted the nMaxLen parameter must be specified. nFound Variable that receives the number of the found strings (optional. nOptions (optional)

WEB_FLAG_CASE_SENSITIVE. If this flag is set the string compare operation is case sensitive. WEB_FLAG_IGNORE_WHITE_SPACE. If this flag is set all white spaces are ignored. WEB_FLAG_DONT_FORCE_LOAD. Specify this option to enable caching for subsequent request. Note that nothing is parsed if the specified table is not loaded (cache hit). WEB_FLAG_SYNCHRON. Parsing operations with this flag set, are done in a sequential way. The second parsing operation is not executed before the first has been completed. WEB_FLAG_ALL_RESPONSES. If this flag is specified all server responses are scanned (even redirection responses, which are normally not displayed by a browser). WEB_FLAG_INCLUDE_EMBEDDED. If this flag is specified, even embedded documents can be specified by the nDocNum parameter. Normally the number of a document is defined by the occurrence of the source definition in an HTML document (src=...). If this flag is specified, every embedded object increases this counter, which assigns higher numbers to subsequent frames. WEB_FLAG_INCLUDE_HEADER. If this flag is specified the response header can also be verified. WEB_FLAG_HEADER_ONLY. If this flag is specified only the response header is verified. WEB_FLAG_RULE. Specify this flag to perform the parsing operation in every subsequent web function. After every web function the specified output variables are set to the new value. If you want to use the variables in other transactions or in event

Silk Performer Courseware

18 of 33

Softsmith Infotech

handler functions use global variables! Call WebCancelAllRules() do disable all verification and parsing rules. Note: The option WEB_FLAG_RULE should only be used in the INIT transaction or in combination with the WebCancelAllRules function! nSkip Specifies the number of parse results, which should not be stored in the array (optional). The first element of the array will be the (nSkip+1) th parse result. Specifies the document to parse (optional). Specify WEB_DOC_ALL if you want to parse all documents. If this parameter is omitted, the first document gets parsed. (see above definition FLAG_INCLUDE_EMBEDDED) Specifies the maximum number of bytes copied to each string (optional). If the sRightBoundary parameter is omitted this parameter must be specified and determines the end of every parsing operation.

nDocNum

nMaxLen

Measuring page times To measure the page times in the log, we can also use following to measure the time Measurestart (StringName) MeasureStop(StringName) MeasureGet MeasureStart
Include file

Kernel.bdh
Syntax

MeasureStart( in sMeasure : string ): boolean;

Return value

true if successful false otherwise

Parameter Description
sMeasure Measure name used to identify the measure when calling additional measure functions and when analyzing the results written to both the results repository and individual result files

Example MeasureStart (<StringName>);

Silk Performer Courseware

19 of 33

Softsmith Infotech

MeasureStop
Include file

Kernel.bdh
Syntax

MeasureStop( in sMeasure

: string

in bIgnoreOnError : boolean optional ): number;

Return value

Final value of the custom time measure in 1/1000 secs.

Parameter

Description

sMeasure Measure name identifying the custom time measure to stop bIgnoreOnError When enabled, measured time is included in calculations if no error has occurred since the timer started (optional). The default value is false. Example MeasureStop(<StringName>) MeasureGet
Include file

Kernel.bdh
Syntax

MeasureGet( in sName : string, in nClass : number, in nKind : number, out fTime : float, in bAll : boolean optional): boolean;
Return value

true if successful false otherwise

Parameter Description
sName Name of the object concerned with the measurement. Must be one of the following:

Silk Performer Courseware

20 of 33

Softsmith Infotech

nClass

Custom time measure. Pass to the function exactly the same name as to the MeasureStart and the MeasureStop function Custom counter. Pass to the function exactly the same name as to the MeasureInc function Transaction. Specify the transaction name as declared in the load testing script. SQL command. Make sure to specify the SQL command identifier (defined in the dclsql section of the test script) in capital letters. Web form. Make sure the specify the Web form identifier (defined in the dclform section of the test script) in capital letters. CORBA object TUXEDO service Specifies the type of measure to retrieve.

To retrieve the value of a custom time measure, pass the MEASURE_TIMER_RESPONSETIME parameter to the function. To retrieve the value of a custom counter, pass the MEASURE_COUNTER_CUSTOMCOUNTER parameter to the function. To retrieve the value of an average counter, pass the MEASURE_COUNTER_AVERAGE parameter to the function. In any other case, pass any of the following parameters to the function, depending on the type of information you are interested in. MEASURE_PAGE_PAGETIME MEASURE_PAGE_PAGEBYTES MEASURE_PAGE_EMBEDDEDBYTES MEASURE_PAGE_DOCDOWNLOAD MEASURE_PAGE_SERVERBUSY MEASURE_IIOP_ROUNDTRIP MEASURE_IIOP_SERVERBUSY MEASURE_SQL_SQLPARSE MEASURE_SQL_SQLEXEC MEASURE_SQL_SQLEXECDIRECT MEASURE_TRANS_TRANSOK MEASURE_TRANS_TRANSERR MEASURE_TRANS_TRANSCA MEASURE_FORM_BYTESSENT MEASURE_FORM_BYTESRECEIVED MEASURE_FORM_HITSOK MEASURE_FORM_HITSERR MEASURE_FORM_ROUNDTRIP MEASURE_FORM_SERVERBUSY MEASURE_TUXEDO_BYTESSENT MEASURE_TUXEDO_BYTESRECEIVED MEASURE_TUXEDO_RESPONSETIME Specifies the type of measure value to retrieve. The following options are

nKind

Silk Performer Courseware

21 of 33

Softsmith Infotech

possible: MEASURE_KIND_SUM. Retrieves the sum of all measure values, for example, the total time for executing all transactions MEASURE_KIND_COUNT. Retrieves the number how often an action was performed, for example, how often a counter was incremented, or how often a transaction was executed MEASURE_KIND_AVERAGE. Returns the average of all measure values, for example, the average time required for a CORBA operation call. MEASURE_KIND_MIN. Returns the minimum of all measure values MEASURE_KIND_MAX. Returns the maximum of all measure values MEASURE_KIND_LAST. Returns the last value set for the measure MEASURE_KIND_SQSUM. Calculates for each measure value the square value, and sums up all square values. MEASURE_KIND_STDEVIATION. Returns the standard deviation of all measured values Variable receiving the measure value Specifies the time interval used for measurement value calculation (optional).

fTime bAll

If this parameter is set to true, the measure value is calculated based on all measurements performed during the simulation. Otherwise, if this parameter is set to false, the measure value is calculated based only on the measurements performed during the measurement interval (default).

Example MeasureGet("<StringName>", MEASURE_TIMER_RESPONSETIME, MEASURE_KIND_SUM, fValue);

Silk Performer Courseware

22 of 33

Softsmith Infotech

WorkLoads

Workload means scheduling the scripts for multiple users. We can schedule the load in following pattern 1. Increasing This is called ramp up where the load to the server is increased by definite time and can run for certain duration called Simulation time. 2. Steady State Here all the virtual users are loaded simultaneously for definite duration

Silk Performer Courseware

23 of 33

Softsmith Infotech

3. Dynamic The load is dynamically used by the system for definite duration

4. Queuing There will be definite wait time between transactions.

Once we set the work load for which the load testing needs to be done, Run the test using the Run test from menu run or

Silk Performer Courseware

24 of 33

Softsmith Infotech

Starting the run test will give the configuration screen for load, we have changed the vusers to 10 and duration to 2:20 min

Silk Performer Courseware

25 of 33

Softsmith Infotech

Distributing users to Multiple IP Address In the above situation, the virtual users are distributed through only one ip address, if we like to distribute the users with multiple IP address then 1. Go to SilkPerformer->Tools->SystemConfigurationManager 2. Select IP address Manager

Silk Performer Courseware

26 of 33

Softsmith Infotech

3. 4. Click Add from IP Addresses

5.

Silk Performer Courseware

27 of 33

Softsmith Infotech

6. 7. Entire list will get generated under IP Addresses

8. 9. you need to restart the system to get the effect

10. 11. The same also can be saved as .smf file which can be loaded for different workspace. Configuring the Servers and its monitors For performance testing, we have to configure the servers to get the information on those servers, The information can be of %cpu usage, Memory available, Concurrent connection etc. To do the same. 1. Select the From SilkPerformer->Results->MonitorServer

Silk Performer Courseware

28 of 33

Softsmith Infotech

2. 3. Select From Monitors->Add data source

4.

5. 6. Select from the system the server you like to monitor

Silk Performer Courseware

29 of 33

Softsmith Infotech

7.

8. 9. Give the connection Parameter

10. 11. Select the parameter which we like to configure

12.

Silk Performer Courseware

30 of 33

Softsmith Infotech

13. 14. If you want to make it as separate series, select No

15. 16. We can add any no. of parameter to this.

Analysis
Analysis can be done based on the Responses for each page and understand the behavior of each page.

By selecting the appropriate measuring units, we can drill down the bottle neck in the pages.

Silk Performer Courseware

31 of 33

Softsmith Infotech

For errors, there is Analyze Error tool which can cumulate all the errors (http errors) and from that we need to provide the solutions.

Actual Base Line Report

From Menu Results Actual Base Line report can be generated. This report is mostly used for every final run.

Silk Performer Courseware

32 of 33

Softsmith Infotech

Silk Performer Courseware

33 of 33

Softsmith Infotech