Scheduling Tests To Run

The ability to run a smoke or regression test at a scheduled time can be a real asset to a Development or IT
organization. Tests can be triggered to run after a daily build is complete or after new content is pushed to a
web application each morning. With SilkTest’s test plan queries, its command line interface and a task
scheduler such as NT’s “at” or the newer “Scheduled Tasks Wizard” , it’s easy to do. Let’s take a closer
look at each of the components.

Testplan Attributes and Queries

SilkTest’s test planning module (a.k.a Silk Organizer) offers the ability to select tests based on an attribute
query. Choose between one of the existing attributes (category, component, developer) or create your own.
To define a new attribute, invoke the Define Attributes dialog by selecting the Main Menu->Testplan-
>Define Attributes menu item. In this example, a new attribute called Environment has been added. The
values web1 thru web5 were also added as possible values for the attribute.

The next step is to label sub-plans, sections or individual tests in your test plan with your new attribute
values. For example, you might have a sub-plan that contains your web environment smoke test. You can
easily select which environments to run upon by labeling your sub-plans by environment and then creating
a test plan query to mark them at runtime. The following is an example of how the test plan might be
labeled to select a smoke test to run in the desired environment(s):

optionset: myapp_web1.opt

environment: web1

include: mysmoketest.pln

optionset: myapp_web2.opt

environment: web2

include: mysmoketest.pln
 optionset: myapp_web3.opt

environment: web3

include: mysmoketest.pln

Using the
test plan
query
feature, a
query is
created to
select each
web

environment. To create a new query, invoke the Mark By Query dialog by selecting the MainMenu-
>Testplan->Mark By Query menu item. Select the attribute(s) and value(s) from the Test Attributes Tab and
name the query for later use.

To mark the tests based on this query, invoke the Mark By Named Query dialog by selecting the
MainMenu->Testplan->Mark By Named Query menu item. This selection provides a means of running
only those tests associated with the attribute(s) specified by the query.
The power of Silk Organizer queries is in the manner in which they can be combined. Through unions and
intersections of different queries, virtually any combination of tests could be selected for runtime. For
example, additional queries can be created by combining the web1 thru web5 queries so that the smoke test
could be run against any combination of environments.

Understanding SilkTest Command Line Options

Now that we have a testplan labeled with attributes and queries to select upon them, the next step is to learn
how to launch SilkTest from the command line. The SilkTest help provides a syntax guide with an
explanation all of the configuration options. For our purposes, only the following configuration options are
required:

partner [[-q] [-query query name [-r] scr.t/suite.s/plan.pln]

The first term in the command line string, “partner”, is the name of the silktest executable. What follows
are two switches and their arguments. The first switch, “-q” allows the user to name the query that will be
used to mark tests for execution. The query name is defined by the user such as “web1” from our example
above. The last switch “-r” indicates what to run. In our example case, a testplan is to be run. You can also
specify the name of a script or suite file. An example of a command line for our example follows:

partner.exe -query web1 –r smoketests.pln

This command line string can be improved by using environment variables to specify the location of the
partner executable and testplan to be run:

"%SEGUE_HOME%\partner.exe" -query web1 -r "%SCRIPTDIR%\ smoketests.pln"

This command line string will run successfully on any machine regardless of the location of the SilkTest
executable or the test plan to be run.

Scheduling Your Test Jobs

There are two ways to schedule your test jobs. If you are using Windows NT and have not installed Internet
Explorer 5.x, your machine is probably using an executable called atsvc.exe as its task scheduler. This
scheduler is also known as “at”. To use at, you’ll want to save your command line as a batch file. To do so,
simply paste your command line string into a notepad file and save it with a .bat extension, e.g.
mysilktestjob.bat. Then complete the following steps:

a. Bring up a command prompt.

b. Type: at time /interactive /every:days “%SCRIPTDIR%\ mysilktestjob.bat”

Where time = the time of day to schedule using a 24 clock and days = the days of the week to
schedule the task using M,F,T,W,Th,F to represent the desired days. For example:

at 06:00 /interactive /every:M,F,T,W,Th,F “%SCRIPTDIR%\mysilktestjob.bat”

c. To check on the status of your scheduled jobs, simply type at without any arguments.

There are some security issues that you may encounter along the way because the at scheduler uses the
system account to launch jobs. For example, if your test attempts to delete browser cookies, you will likely
not have adequate permissions. To resolve this problem, change the location where the browser stores
temporary internet files from the default of “all users” to a location where the current user has read and
write permissions, e.g. “C:\WinNT\Profiles\youraccount\Temporary Internet Files”. Another limitation is
that you will not be able to copy files to other machines on the network because the network does not
recognize the system account on the local machine. Although Microsoft’s documentation suggests that you
can configure the at scheduler to use an alternate account with adequate permissions, the facility appears to
be disabled.

If you are using Windows2000 or Windows NT and have installed Internet Explorer 5.x your machine is
probably using an executable called mstask.exe as its task scheduler. This executable provides a mechanism
for using an alternate account to resolve the permissions issues described above. It also provides a handy
wizard for scheduling your tasks. To schedule your test job, simply complete the following steps:

a. Doubleclick MyComputer on the desktop

b. Doubleclick on Scheduled Tasks
c. Doubleclick on Add Scheduled Task
d. Follow thru the steps of the wizard:

- Select SilkTest from the list of applications

- Select Daily

- Select Weekdays unless you want the task to run on the weekend as well

- Select the desired start time of the task

- Enter the user id and password of the user to run the task as (typically your own)

- Check the CheckBox to invoke the advanced properties dialog

- Complete the Run field per this example: "%SEGUE_HOME%\partner.exe" -query web1
-r “scriptdir location on your machine\smoketests.pln"

- Complete the Start in field as follows: % SEGUE_HOME %

The next time you get home and realize that you forgot to kick off the daily regression, consider this
automatic scheduling alternative. It could save you a day on your schedule.