BASIC USER TRAINING PROGRAM Module 7: Variables, Parameters and Field Substitution
Objective
You will have an understanding of how to use variables, parameters, and field substitution to make test assets more portable and reusable.
Outline
• • • • • • •
Field replacements (substitution) Variables Parameters
Storing and querying on response data Saving data from a response Lab Quiz
FIELD REPLACEMENTS (SUBSTITUTION)
Just before executing a step, iTest evaluates field replacement text and replaces the text with the values of a variable or a parameter or a value returned in a response. This process, often called runtime substitution, enables you to insert variables into commands, property settings, Messages and most places that use text.
Before we talk about adding the variable, parameters into the Test Case, let’s talk about the concept of substitution. Substitution can make your test case more portable and more readable.
In this case, it allows us to automate the dynamic data. For example, as you remember in the previous
module, we have not created a process to automate checking whether the ping count is increasing properly. Because count is a dynamic piece of data, we can actually use substitution by storing ping count in a variable, and then checking the value of that variable later. But if you just think about substitution in general, this test case could apply to several different devices, and here I’m pinging a specific device with a specific ping count. Well if I wanted to make this test case more portable or more usable, I could actually put in more values here then I could substitute at run times then I could run this test case against many different hosts, or with a different ping count.
So as a different example, select step for command show ip int Vlan1. We can have more than one interface. So what if I actually wanted to replace vlan1 with something that could be more flexible, something that I could put different values in rather than putting in each individual step So if you remember from a previous module we mentioned that the three columns are just properties from that step there are several properties attached of that step which you need to access via the step properties menu. So I’ll click step properties, I’ll scroll here to adjust the screen so here are step properties, you should remember this from an earlier module, remember the grid here shows us this information and there are several other properties over here.
Almost all property fields in iTest support substitution. Each property in iTest that can take input and support substitution has a small decorator located to the left of the field. This decorator will either be a solid gray box indicating that substitution is disabled for the property field, or it will be a gray box with a white F visible indicating that substitution is enabled for the property field.
iTest displays an icon on the text box for properties that will undergo substitution of field replacement text at runtime. In the example, you can see that the Thread name, Session, Context, and Command properties will be substituted at runtime if the property string includes field replacement text. Notice that theindicates that iTest will not perform substitution for the Target text. The indicator appears if you select multiple steps in the Test Case editor and the steps have different settings.
Here you can see two property fields. The Target field has substitution disabled and the Command field has it enabled.
To enable or disable substitution, click the decorator.
Certain fields support substitution and you actually cannot disable substitution for the field. To determine the options, hover the cursor on the decorator. So for instance in the session property, iTest is telling us that it supports field replacement and we see that with the “f” but field replacement cannot be disabled on this field. Now notice that the action doesn’t support the field replacement. You cannot do substitution in here. The reason being is that it is a specific number of actions that substitution here wouldn’t make sense.
Context and target we haven’t really covered here yet these relate more to our GUI sessions. But as you can see they both do support field replacement. Context has it turned on by default, target doesn’t. And so notice that we have different messages.
To toggle whether or not field substitution is enabled is to click the box. I just left clicked and now notice the context field decorator is now just a grey box. And now notice you also get a star icon on the Test Case tab. Here indicating that my test case has changed. I go back to context, and I click again and the f comes back. Now in the command field it’s a bit of a different case. We have to use this check box as well. For test steps specifically there is a special ability to allow or disallow field substitutions for the command field. This checkbox property can be located towards the bottom of the General step properties page.
If you toggle this checkbox you will notice that the decorator for the command field disappears or reappears for the Command property field. This may seem redundant as you can use the decorator to enable and disable substitution but there are two reasons for this checkbox being explicitly called out.
1. As the Command field is the most interfaced with field in iTest and the field most likely to contain
substitution, this explicitly calls out the current substitutive state to the user (even if they are unfamiliar with decorators, which are not obvious to new users).
2. More importantly, there are circumstances in iTest where you can’t disable field substitution, must
disable it, or should disable it. This checkbox allows iTest to explicitly change the substitutive state of the field based on the configuration of the test case (as the user inserts steps, makes action selections, etc.). For certain steps like for loop steps (loops will be covered under condition logic in the next module) you may notice the checkbox is disabled, indicating you can’t (even if you attempt to toggle the decorator) disable field substitutions for this type of step. Disabling substitutions for a “for” loop step would cause it to automatically fail to be processed properly.
Check the box if the string specified for the Command property uses a command field replacement, a variable, or a backslash that is used to escape a special character. As a result, the substitutions will be performed before the system uses the text that appears in the Command field Field replacement is the ability to do substitution in that field ability, for example to take Vlan1 and make it a variable or make it parameter. And as we get a little further in this module, we’ll show you the syntax of a variables and parameters. For a variable, place a dollar sign in front of the name of the variable to mean \"the value of the variable.
So the boxes indicate that the field supports substitution. The “f” in the box indicates that it’s turned on. So what that means, is that iTest will actually do that substitution that I just mentioned, if there was no “f” in the box, then iTest would literally show ip int Vlan1 and whatever the name of the variable is to the device. Types of substitution
There are many types of substitution in iTest. There is nothing to prevent an experienced user from just typing the substitution by hand. However, the easiest way to insert substitution syntax is to use the Insert menu item on the right-click context menu that appears for fields that support input.
The three most common substitutions are variable, parameter, and query on stored responses. These types are described in later sections of this module. You can also read additional information on the different types of substitution in the iTest online help.
So let’s add some substitution, variables and parameters.
In the previous module, we created and executed the Test Case module6_testcase.fftc.Before we get into adding Analysis and Response analysis to the test case, let's save a copy of the test case as module7_testcase.fftc so that we can build on top of it.
1. Click the “Home” icon to return to the home Activities page. 2. Click Manage Workspace
3. Go to my_project and create a new folder under test_cases called Module 7--Variables, Parameters, Substitution
4. Select module6_testcase.fftc under Module 6—Response Analysis
5. Click the Test Case tab in the Test Case Editor to make sure that it has the focus
6. Click File > Save as > Select my_project/test_cases/Module 7--Variables, Parameters, Substitution and name it module7_testcase.fftc
The Test Case should look like this:
So just to review, variables can be modified during test case execution. A variable stores a value and can be modified during test case execution. So for example it could be set up here, used, reset, used, reset, used again etc.
A variable stores a value that can be modified during test case execution.
A parameter value is set before execution begins; you set the value of a parameter and then run the test. Once a test starts, the value of the parameter cannot change.
VARIABLES
So getting back to variables, there are two different ways you can set a variable in iTest,
1. Set it directly in the test case
2. Pull a value out of the response, and set it through an analysis rule.
Let’s look at examples of both. Setting a variable using an Eval action
We will now create a variable that can be used as a comparator for analysis on the ping –c 5 ffciso1.fnfr.com steps.
1. Select the first row in the test case, labeled procedure in the Action column. 2. Use Ctrl+Enter to insert a new step.
3. Select the action as Comment put in “set a variable for the ping count”.
4. Insert another step. Instead of comment now, select the action called eval. Type eval in the action field. It will show the help for the description on eval action. As it says “eval evaluates the statements specified in the Description cell”
The eval action allows us to act on the iTest interpreter. What is the iTest interpreter? The iTest interpreter interprets the commands in your test case and setting and reading variables is done by the interpreter. Evaluating loops or if-then statements done by the interpreter, and those are things we’ll look at in the later module. If you are a programmer, then the concept of an interpreter should be familiar to you.
5. You set a variable value by using the “set” syntax. If you type “set” you see “set variable name value”. So that’s just giving a clue to the syntax. The name of the variable is ping_count and let’s give it a value of 6. Now notice my red “x” goes away. The reason the red “x” was there was because I had an empty eval statement. So again that’s just the iTest real-time error checking.
5. Press the tab key three times to commit the changes and move to the Description field. 6. Enter set ping_count 6 in the Description column and press Enter to commit the changes.
NOTE: When editing a field in iTest, the changes you have made are not committed until you leave the field or press Enter. You can also use the Escape key to undo any changes you have made to the field.
7. Select the ping –c 5 ffcisco1.fnfr.com command. We want do a substitution on 5 here. We will put in the name of the variable and then have iTest substitute the value of the variable that we have set in the previous step when we run.
So now look at this step properties on command field, the box with the “f” means that it supports field replacement.
8. You can remove the 5 and type in here the correct syntax to call the variable. And as you get more familiar with iTest you’ll come to figure out that it will go straight to that step.
What you can also do is select 5, right click and select Insert. And notice that we now have a choice for a local variable. Now we have two choices, Get using $ syntax and Get using command. (Remember that we are getting the value of the variable and substituting it into this command.) Let's use Get using command. iTest inserts “get” and the variable name to replace the hard-coded value '5' with the a get of the variable value. Now notice also that we have these square brackets. They indicate that the iTest, interpreter should evaluate this statement first (before processing the step), so it’s going to get the value first, and then execute the ping-c command with whatever the value evaluates to ffcisco1.
9. Our other choice is to get the variable value using $ syntax. Here's what that looks like: We see here we have $, curly brace and then variable name. Replace the word variable_name with the variable name that we set earlier: ping_count
10. Select the comment that appears before the ping command. Here, it says – use the linux box to send 5 pings to the router. Open up the properties and you will notice that you can do substitution in here even though it’s a comment. So I could actually put ${ping_count} in the comment.
11. Save the test case and execute it. Open the Test Report and you will notice the substitution has happened.
Setting variables using analysis rules
Let's take look at the Test Report and see what actually executes:
• Display the ICMP received echo counter on the router • Use the Linux box to send 6 pings to the router
• Recheck the ICMP received echo counter on the router to ensure the count has increased appropriately.
Well we can actually use a combination of variables here to make the test dynamic. So the next time it runs through, it doesn’t matter what the ICMP echo counter is in the first test, we will just store it in an iTest variable on first show ip traffic command and remember it. Then, when we check it on the second show ip traffic command, we will sure that whatever the counter equals at that point, is equal to whatever we stored here, plus the number of pings that we set. Let’s go ahead and do that.
So the test here is to check the ICMP Rcvd count after we ping the device. We want to make sure that the count is increased by the same count as the number of pings sent. Generally, we do this by looking at the response and check the ICMP Rcvd count.
1. Execute module7_testcase.fftc. Select the ping command.
2. Locate the 5 packets transmitted, 0% packet loss and time 4003ms in the Response view.
--- ffcisco1.fnfr.com ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4003ms rtt min/avg/max/mdev = 1.138/1.749/2.463/0.518 ms, pipe 2 7. 3. Add the analysis rules using Quick Analysis Rule > Regular expression > Store the extracted data in a variable from the Response view.
Now we want to store three values into variables:
a. Store 5 in “packets_transmitted” b. Store 4003ms in “time” c. Store 0% in “packet_loss”
The analysis rule looks like this: 4. Select the first “show ip traffic” step and locate the ICMP received echo counter in the Response view. ICMP statistics: Rcvd: 0 format errors, 0 checksum errors, 0 redirects, 56978 unreachable
13913 echo, 110 echo reply, 0 mask requests, 0 mask replies, 0 quench
0 parameter, 0 timestamp, 0 info request, 0 other
0 irdp solicitations, 0 irdp advertisements
5. Add an analysis rule using Quick Analysis Rule > Regular expression > Store the extracted data in a variable from the Response view and store 13913 in the ‘icmp_rcvd_echo_before’ variable.
The analysis rule looks like this: 6. Select Second “show ip traffic” step and locate the ICMP received echo counter in the Response view. Here is the sample output (ICMP statistics) from “show ip traffic” on Second show ip traffic
ICMP statistics: Rcvd: 0 format errors, 0 checksum errors, 0 redirects, 56978 unreachable 13918 echo, 110 echo reply, 0 mask requests, 0 mask replies, 0 quench 0 parameter, 0 timestamp, 0 info request, 0 other 0 irdp solicitations, 0 irdp advertisements
7. Add an analysis rule using Quick Analysis Rule > Regular expression > Store the extracted data in a variable from the Response view and store 13918 at ‘icmp_rcvd_echo_after’ at ‘icmp_rcvd_echo_after’
The analysis rule looks like this: 8. Use Ctrl+Enter to insert two new comment steps before the exit steps.
We will be using comments to “message out” to the Execution view. Make sure to go to the Test Case editor General page and select 'Generate an execution issue' for every comment step executed. Tip: Comments can include field replacements. This is an easy way to send data to the Execution view.
9. Add the following message in the Description field of the first comment step:
Sent $packets_transmitted packets in $time with $packet_loss% packet loss.\\n
10. Add the following message in the Description field of the second comment step:
ICMP Rcvd Echo went from $icmp_rcvd_echo_before to $icmp_rcvd_echo_after (a difference of [expr ($icmp_rcvd_echo_before + 5) - ($icmp_rcvd_echo_before)] packets).
13. Execute the test case. You can see the results in the Execution view.
PARAMETERS
Parameter values are specified before execution and cannot be altered during test case execution. Parameters can be defined in test cases, session profiles, testbed and global parameter files. Session profile parameters are inserted into test cases using a syntax related to session parameters. Test case and global parameters are merged together (if a testbed is in use) to create a single list of available parameters. You only need to worry about parameter merging if you have a parameter with the same name that exists in both the test case and the session profile being used by the test case or if there is a duplicate parameter name in the global parameter file (if being used).
Topology files do not have parameters. Instead, topology files have the notion of properties. We will cover that later.
Let’s update our test case to make use of a test case parameter.
Select the Parameters tab (located at the bottom of the Test Case editor). (The Session Profile editor also has a Parameters tab, for example, open the cisco_telnet_reference.ffsp session profile and you can click the Parameter tab.
To create parameters
1. Open the iTest Explorer view in the ‘Develop the test case’ activity page by clicking Windows > Show View > iTest Explorer
3. In my_project, click File > New and create a new folder called parameter_files
4. Click File > New to create a file called cbt_parameters.ffpt in the parameter_files folder.
5. Click Finish. You can see that this is just the file where you can save the parameters. This is similar to what we have in test case or in session profiles.
6. Only one parameter file is used by GUI iTest. So to tell iTest which parameter file to use, you have to make it global. Right-click the file and select “Set As Global Parameter File”. When running from the command line, you do not need to set the parameter file as global. You can tell iTest which file to use during runtime. So keep that restriction if you want to decide where you want to keep your
parameters. We will not use the parameter file but we will create one parameter to show you how the parameters get merged later.
7. Click the button to add a new parameter. You see a param1 has been added. Double click the name of parameter param1 and rename it to test_parameter. Give the new parameter a value of set_in_the_param_file. Enter the description as This parameter is set in the parameter file
Tip: if you are setting up a password, check Mask the value.
You can add parameters in a hierarchy by adding a child node under an existing node in the parameter tree. This does not affect how the parameters are called as it is only used to organize your parameters for viewing.
8. Save the file. Now let’s go back to the module7_testcase.fftc in the Test Case editor.
9. Let’s set the ping target as the parameter
10. Go to parameter tab in the Test Case and click the Add a new parameter button. Double-click the name of the parameter param1 and rename it to ping_target. Enter the value: ffciso1.fnfr.com
Enter the description “The target of the ping test”
13. Go back to the test case steps, select the ping step and highlight ffcisco1.fnfr.com and remove it.
14. Right-click and select ‘Parameter’ from the menu
15. The “Insert Parameter” dialog box opens. You will notice that we have two choices for Parameter source-
Testcase/Testbed/Global Parameters Sessions
Two parameters are available (they were defined in the different places). This is the result of merging
16. Select ‘ping_target' and you will notice that in the test case ‘ffcisco1.fnfr.com’ has been substituted by a parameter. The syntax is [param 18. Open the test report and notice the parameter substitution has happened on the ping step. Topology and parameters As we mentioned earlier, topology files do not use parameters. Instead, topology files have the notion of properties. 1. When you have a Topology editor open, click any white space and go to the properties view and select the topology tab. Here you can add properties and nested properties in property collections. 2. Set a property called packet_loss at the Topology level Properties are NOT accessed using the param command. Instead, you access a property using the [tbml property] command. 3. At this point you have created the parameter. Return to the Steps tab of the Test Case editor. 4. We will use the newly created parameter to analyze the ping command response to validate no packets have been lost during the ping. Select the ping step in the test case grid. 5. Locate the packets lost value in the Response view. Add an analysis rule to the counter for packet loss (Quick Analysis Rule > Regular expression > Compare and take action accordingly). 6. Highlight the “0” value and replace it with [tbml property ping_target] If you have parameters attached to a session profile that is, in turn, attached to a resource in a topology, you continue to access those \"profile parameters\" using the [profile] command. For example if you define a sp_ping_target parameter in the ‘linux_ssh_reference.ffsp’ base session profile. You will have the following syntax when using it in the test case: [profile linux_box_sp_ping_target] So at runtime, the test case can use parameter values from several sources: the testbed file, session profiles, and the parameters that you configured on the test case’s Parameters page. You can also specify a Global parameter file so that any executing test case uses the parameters defined in the file. In addition, if the Global parameter file references other parameter files, then the parameter definitions in the referenced files are included. Because you can combine parameter settings from so many locations, you can implement quite sophisticated dynamic operation. iTest combines the parameter definitions from all sources before execution. The goal is to give the user the way to see how the parameters from multiples sources will aggregated at runtime. Parameter editing is possible at five locations in iTest. 1. 2. 3. 4. For Parameter file on the parameter page For Testbed file on the parameter page For Session Profile on the parameter page Finally, parameter page of Test Editor STORING AND QUERYING ON RESPONSE DATA a. Storing a response into a variable (for use later in the test) You can store the response to any step in a variable. By doing so you make it so any step after the stored step can be referenced to extract data from its response. This allows you to do comparative analysis between steps, leverage response data for commands, and so on. You can set a step property to cause iTest to store the full response for a step into a variable. A subsequent step can make use of the response data by applying a response or query command to the data in the variable. You have the following options for storing and accessing response data: 1. By default, the variable is a complex object that includes both the text of the response and the structured data part of the response. To access the contents of the variable (in a subsequent step): • Use a response command to return all or part of the response’s text content. • Use a query command to return the result of a query against the structured part of the response. 2. If you set the Store only the text of the response property for a step, then only the response text (and not the structured data part of the response) is stored in the variable. To access the contents of the variable (in a subsequent step): • Use $varName or [get varName] (or, for global variables, [gget varName] or ${/data/varName}) to return the full text of the response. b. Storing an entire response in a variable We can store the entire response of the command into a variable and use it in a later step. 1. Click the show ip int brief command 2. See the response of the command and notice the blue boxes indicating that a response map has already been created 3. In the Step Properties section, navigate to “Other Post-processing” and then click Store Response4. Enter the name of the variable in the “Store the response in variable” field. The best practice is to name it with the name of the command plus \"_response\". So let’s name it “show_ip_int_brief_response” You will notice the pencil icon on the step and the step property. If you hover over the icon, it will indicate that Property storeResponseAt has non-default value. 5. Insert a new step underneath show ip int brief and select eval action 6. for the eval command, enter puts and you see the hint as put string 7. This command puts the string from the Response view. The command will be puts $show_ip_in_brief_response 9. Execute the test case. 10. Open the Test Report and notice that the puts command has the complete response from the show ip in brief command. When iTest stores the response in a variable, it actually stores more than just the text of the response, it also includes the queries. So, to get at the structure information, you cannot just use the $ and variable name. You need a special syntax. Right-click and then select Insert > Stored Response > local iTest inserts [response So the command will look like puts [response show_ip_int_response] 12. Execute the test case. This time you will get only the response in the Response view. We will not get the blue boxed because the response map is being applied through sessions. We will discuss this in the response map module. QUERY ON STORED RESPONSE Example1 As you notice, on the show_ip_int_brief command, Queries are saved and the blue boxes indicate that response map is applied and it is safe to use queries 1. Let’s add another eval step and enter another puts command. 2. Right-click and select Query on Stored Response. 3. You will get the option to get the queries from the current step as well as the above step – show ip int brief. There are no queries available on the current step and all the queries available to you for the show ip int brief 4. Select Status_by_Interface (“FastEthernet0”) 5. Execute the test case and you will notice that you get the result of the query in the response view 6. You can edit the query in the step, for example, replace the 0 with 1 to get the Status by FastEthernet 1 as “administratively down” 7. Execute the test case and you will get the status of interface 1 as “up” 8. Let’s add a comment step above the puts steps to improve readability of the test case: “Execute a few examples using a stored response variable” Example2 Let’s leverage this type of substitution in our current test case to intelligently analyze that the echo counter increases appropriately based on the initial echo value (the first show ip traffic command) and the number of successful packets received from the ping command. 1. Select the first show ip traffic step. 2. If you look at the Response view you will notice that there are blue boxes drawn around several areas of the response. Queries are the identified fields of data from the response. A Query on Stored Response requires that the stored step has queries generated. To see if the step has queries, select the Queries view. 3. Drag the Queries view and dock it on the upper-right corner of the window. 3. Locate the query that identifies the received packets value in the Queries view and select the ip_rcvd_total() entry. 4. Notice that the value in the Response view is highlighted to indicate what data from the response is identified by the selected query. We now know that the query we will use for analysis later in the test case is called ip_rcvd_total() 5. Expand the step properties for the first show ip traffic step and select the Other Post-processing > Store Response property page. Enter show_ip_traffic into the Store the response in variable field. 6. Select the ping step and locate the query that identifies the received packets value. 7. Minimize the view group that contains the Queries view. 8. The step properties should be open and the Store Response property page displayed. Enter ping into the Store the response in variable field. 9. Minimize the Step Properties section of the test case editor. 10. Select the second show ip traffic step and locate the ICMP received echo counter in the Response view. 11. Add an analysis rule (Quick Analysis Rule > Regular expression > Compare and take action accordingly) from the Response view. 12. Highlight the “1197” text (or the value displayed for your device) and right-click the highlighted area. Select Insert > Query on Stored Response menu item. This opens the Insert Query on Stored Response dialog. In the Stored Responses area select the show_ip_traffic entry in the Variable Name column. Locate and select the ip_rcvd_total() entry in the Response Queries list. 7. Click the Insert button to insert the substitution syntax into the assert field. After the inserted syntax and a space, + sign and another space. 8. Ensure you are in edit mode on the assert field and that the cursor is at the far right of the cell. Right-click and select Insert > Query on Stored Response menu item. This opens the Insert Query on Stored Response dialog. In the Stored Responses area select the ping entry in the Variable Name column. Locate and select the Received () entry in the Response Queries list. Click the Insert button. 9. Press Enter to commit the changes to the field. 10. Execute the test case in a new window. 11. The Execution view should display a pass issue for the newly added analysis rule. 12. Close the execution view and return to the main iTest window. NOTE: Avoid using a variable name for the stored response that is the same as a query name from the stored response. Lab 1. Experiment with adding variables to your test case using both “eval” and Analysis Rules 2. Save the IP Address of Vlan1 in a variable. This can be done in the “show ip int brief” step 3. Insert a step after the second “show ip traffic” step to ping the IP Address of Vlan1 (using the variable). Send 3 pings more than defined in “$ping_count,” but don’t define a new variable HINT: Add 3 to $ping_count in the command. Look up “expr” in the help 因篇幅问题不能全部显示，请点此查看更多更全内容