Documentos de Académico
Documentos de Profesional
Documentos de Cultura
Tes
BE TIC T Ap
AC NE
ST ES ps
t
:
VOLUME 6 • ISSUE 4 • APRIL 2009 • $8.95 • www.stpcollaborative.com
Speak in Tongues:
The CERT C Spec
S ni f f O u t S e c u r i t y
F l a ws W i t ho u t A
F au s t i a n B a r g a i n
Contents A Publication
12
COV ER STORY
Making a Career of Evil—Using A
Hacker's Tool to Secure Your Apps
Fuzz testing turns the tables on those that would do harm. Learn about
this negative testing technique that takes penetration to a whole new level.
By Ari Takanen
Speak Security
16 Lingua Franca
Make CERT C your native tongue and
build secure applications from the start.
Developed by Carnegie Mellon Univer-
sity, the specification translates or
dinary C-language code into safe and
reliable. By Paul Humphries
9 • ST&Pedia
Industry lingo that gets you up to speed.
Bad Choice vs.
29 Worse Choice
10 • The Conference Report
Here's what you missed at February's Future
Test conference in NYC.
Delay release or deploy with bugs? When 33 • Best Practices
your only two options are bad and worse, Such fragile creatures, those .NET applica-
there's sometimes another way to go. tions. By Joel Shore
By Matt Love 34 • Future Test
The difference between traditional and mile-
stone consulting. By Phil Simon
Software Test & Performance (ISSN- #1548-3460) is published monthly by Redwood Collaborative Media, 105 Maxess Avenue, Suite 207, Melville, NY, 11747. Periodicals postage paid at Huntington, NY and additional mailing offices. Software Test & Performance is
a registered trademark of Redwood Collaborative Media. All contents copyrighted © 2009 Redwood Collaborative Media. All rights reserved. The price of a one year subscription is US $49.95, $69.95 in Canada, $99.95 elsewhere. POSTMASTER: Send changes of address
to Software Test & Performance, 105 Maxess Road, Suite 207, Melville, NY 11747. Software Test & Performance Subscribers Services may be reached at stpmag@halldata.com or by calling 1-847-763-1958.
•
kmuns@stpcollaborative.com
companies,” according to its part of your routine; ITToolbox
About page. When I first started (12 percent), Open Source Reprints
receiving invitations to join Testing (11 percent), DZone (5 Lisa Abelson
social networks years ago, I resis- percent) and Daniweb (2 per- abelson@stpcollaborative.com
ted, even though most were Most testers cent). But 41 percent replied that (516) 379-7097
from people I knew and trusted. none of those listed were “online
My thinking was that I had use LinkedIn forums you visited most.” Subscriptions/Customer Service
enough on my plate just to main- Of this latter group, the stand- stpmag@halldata.com
tain my own contact database regularly; about out was opensourcetesting.org, a 847-763-1958
(now 4,500 strong). Why should Web site devoted to software test-
I help maintain someone else’s a third visit ing tools. It was created and is run Circulation and List Services
professional contact list? But Ted by Mark Aberdour in his spare Lisa Fiske
Bahr, my boss at the time, urged Facebook, time. Aberdour is CEO of Kineo lfiske@stpcollaborative.com
me to give it a try. Open Source, a solution and serv-
What I realized after joining QAforums or ice provider based in the U.K.
Cover Illustration by Misha
was that these networks provide The not-for-profit Web site does
a great way to stay in touch with Stickyminds. offer a discussion forum, but it’s
contacts long since forgotten. (I lightly trafficked; the site’s main
•
also learned that one should not thrust is as a repository. As such,
simply invite everyone in one’s it does the job well; it’s informa-
database, as many have forgotten tive, well organized, and extreme-
me). The main takeaway is that ly well stocked.
my 548 direct connections link With the dearth of busy
me to more than five million other people, any forums out there, clearly software test and QA
President Chairman
of whom I can communicate with through a professionals need more places to interact. Andrew Muns Ron Muns
number of channels. Which ones do you use and what would you like
105 Maxess Road, Suite 207
Back to our recent survey: Nearly two in five to see done differently? If you haven’t taken our Melville, NY 11747
(38 percent) of you use Facebook, which in my one-page survey yet, please take two minutes +1-631-393-6051
fax +1-631-393-6057
experience is used for personal contact far more now and visit tinyurl.com/cmtkqt. We look for-
ward to hearing from you. ý
www.stpcollaborative.com
than for business. Its interface appears to be
You’ve heard of fuzzy math. If you turn to page 12, you’ll learn about fuzzy testing, a practice with
roots in the world of hackers. ARI TAKANEN, chief technical officer at Codenomicon, tackles the sub-
ject of our lead feature, explaining fuzz testing’s usage scenarios beyond that of penetration testing and
security auditing. Ari is a noted speaker and author on software testing and security. He conducted exten-
sive research on fuzz testing with the Oulu University Secure Programming Group, and also was involved
in the pioneering work done by the PROTOS project (1998 to 2001).
PA U L HUMPHREYS is a software engineer with LDRA Ltd., and responsible for ongoing enhancement
of LDRAs’ static code analyzer. LDRA provides solutions and services for safety-critical systems in aero-
space, defense and other industries. A veteran of software development for nearly two decades, Paul has
been with companies such as British Aerospace and GEC Marconi. He holds a masters degree in Computing
for Commerce and Industry. Beginning on page 16, Paul explains best practices for producing reliable
and secure software systems using CERT C, a secure language developed by Carnegie Mellon's Software
Engineering Institute.
One key problem with security code audits is that they tend to cause more problems than they solve.
Beginning on page 29, M ATT LOVE, a software development manager at test tools maker Parasoft, helps
you solve the “one size fits all” problem of having to decide between delaying the project or going to
market as-is. Matt has been a Java developer since 1997. He holds a bachelor’s degree in computer engi-
neering from the University of California at San Diego.
Index to Advertisers
Advertiser URL Page Number
Hewlett-Packard www.hp.com/go/alm 36
Klocwork www.klocwork.com 28
Wildbit www.beanstalkapp.com 2
SaaSy Services
“There’s a drive toward refreshing HP has modified its support and consul-
Among Performance Center 9.5's latest fea-
existing apps to make them more inter- tures is Trending, which automates the job of ana-
tancy policies to better suit short term
active and engaging,” said Subbu Iyer, lyzing performance data from one release to the projects, now offering 1- and 3-month
senior director of products for HP next, presenting stats graphically in a browser. engagements in addition to the previous
Software and Solutions, of the move to minimum of one year. “If you need
so-called Web 2.0 standards. “That intro- tems in a high-performance way.” resources, want to leverage our skills for
duces a slew of performance issues. For For situations in which the tester testing oracle apps, let’s say, or your work-
Ajax, there are several frameworks,” he might not know which protocols are in load got higher but they don’t want to
said, for example, and referred to the use, Performance Center 9.5 introduces hire, we have the ability to do the testing
asynchronous nature of the technique. Protocol Advisor. “Developers often miss for you for a short term.” Performance
“Ajax-related architecture [also] intro- out on telling testers all the protocols or Center 9.5 and the short-term services
duces performance issues.” technology that’s embedded in the appli- became available on Feb. 24.
Industry's Best
Gather in NYC For
Web-Test Confab
By Edward J. Correia
Negative Requirements
To understand the principles behind fuzzing, it’s helpful to look at how it fits into the
entire software lifecycle. Since the software development process starts from require-
ments gathering, let’s first look at how the requirements for security and fuzzing can
be mapped together. A software requirement specification often consists of two dif-
ferent types of requirements. First there’s a set of positive requirements that define
how software should function. Then there’s the negative requirements that define
what software should not do. The actual resulting software is a cross-section of both.
Acquired features and conformance flaws map against the positive requirements.
Fatal features and unwanted features map into the negative requirements. The unde-
fined grey area between the posi-
tive and negative requirements
leave room for the innovative fea-
tures that never made it to the
requirements specifications or to
the design specifications but were
implemented as later decisions
during the development. These are
often difficult to test, and might
not make it to the test plans at all.
The main focus of fuzzing is not to
validate correct behavior of the
software but to explore the nega-
tive requirements.
In its simplest form, mutation fuzzing can be accomplished with bit flipping, data
insertion or other random data modifications. The idea is to try unexpected inputs.
The other fuzzing method involves building a model from communication protocol
specifications and state-diagrams.
Ari Takanen is chief technical officer at Codenomicon, which makes tools for testing software security.
www.stpcollaborative.com • 13
FUZZ TESTING
Fuzz Buzz
The purpose of fuzzing is to find securi-
ty-critical flaws. The timing of such tests
will have heavy impact on the total cost
of the software. Therefore the most com-
mon view in analyzing fuzzing benefits is
to look at costs related to identification
and repair or security-related bugs.
Software security has a special additional
attribute to it, as most of the costs are
actually borne by the end user in the
form of maintenance, patch deployment
and damages from incidents.
Security compromises or denial of
service attacks impact the users of the
software, not the developers. This is why
the cost metrics often include the repair
costs for the developers as well as the
costs from damages to end-users. These
are often the very same metrics that you
Mutation-based fuzzers break down is the traditional approach of validating might have developed for analyzing the
the structures used in the message and verifying functionality. Perfor- needs for static analysis tools. The cost
exchanges and tag those building blocks mance testing looks at the efficiency of per bug will vary depending on which
with meta-data that helps the mutation the built system. Both exercise the sys- phase of the software lifecycle your test-
process. Similarly, in full model-based tem using valid inputs. ing efforts take place in (the earlier the
fuzzers, each data element needs to be Introduced by PROTOS protocol- better). This type of analysis is not easy
identified, but that process also can be security researchers in 1999, robustness for static analysis tools due to the rate of
automated. The information needed is testing on the other hand, looks at the false positives that do not have any signif-
often already given in the specifications system under invalid inputs, and focuses icance for security. A metric collected
that are used to generate the models on system stability, security and reliabili- early in the process might not give any
(Figure 1). ty. By comparing these three testing cat- indication of the real cost savings.
Besides information on the data egories, we can note that most feature It’s different for fuzzing. While a stat-
structures, the added meta-data also can tests map one-to-one against use-cases ic analysis tool often delivers a poor suc-
include details such as the boundary in the software specifications. Perfor- cess rate based on analyzing the real
limits for the data elements. In model- mance testing however, uses just one security impact of the found flaws, with
based fuzzing, the test generation is use-case but loops that in either a fast fuzz testing there are no false positives.
often systematic, and involves no ran- loop or in multiple parallel executions. All found issues are real and will provide
domness at all. Although many muta- In robustness testing, you build thou- a solid metric for product security
tion and block-based fuzzers often claim sands or sometimes millions of misuse- improvements.
to be model-based, a true model-based cases for each use-case. Fuzzing is one
fuzzer is based on a dynamic model that form of robustness testing, focusing on Fuzz-Test Automation
is “executed” either at runtime or off- the communication interfaces and dis- Fuzzing maps nicely to various test
line. In PROTOS research papers, this
approach of running a model during
the test generation or test execution was
called Mini-Simulation. The resulting
executable model is basically a full
implementation of one of the end-
W ASN’T FUZZY, WAS HE?
The term ‘fuzzing’ or ‘fuzz testing’ emerged around 1990, but in its original meaning
fuzzing was just another name for random testing, with very little use in QA beyond some
limited ad-hoc testing. Still, the transition to integrating the approach into software
points in the communication. development was evident even back then. From 1998 to 2001 the PROTOS project (at
University of Oulu) conducted research that had a focus on new model-based test
Fuzzing Among Other Techniques automation techniques as well as other next-generation fuzzing techniques. The purpose
Looking at different types of black-box was to enable the software industry itself to find security-critical problems in a wide
testing, we can identify three main cate-
range of communication products, and not to just depend on vulnerability disclosures
gories of testing techniques. These are
from third parties.
feature testing, performance testing
and robustness testing. Feature testing
W
automation techniques. While differ-
ent levels of test automation are used HERE'S THE FUZZ?
in all testing organizations, fuzzing
can be added just about anywhere in While fuzzing was originally intended as a tool mainly for penetration testers and secu-
the domain. In fact, test automation rity auditors, today its usage is more widespread and diverse. Soon after the exposure
experts are often the first people that caused by PROTOS, fuzzing quickly became adopted by network equipment manufactur-
familiarize themselves with fuzzing ers for their quality assurance processes. From that, fuzzing technologies evolved into
and other related test generation tech- quality metrics for monitoring the product lifecycle and product maturity.
niques. Test automation often focuses
only on the repeatability of tests. But Perhaps because of the rapid quality improvements in network products, fuzzing soon
automation has led to significant also became a recommended purchase criterion for enterprises and pushed by vendors
improvements in test design and effi- who were already conducting fuzzing and thought that it would give them a competitive
ciency. edge. As a result, service providers and large enterprises started to require fuzzing and
The more advanced your tools, the similar testing techniques from all their vendors, further increasing the usage of fuzzing.
less work that will be required to inte- Today fuzzing is used in three phases at the software lifecycle:
grate fuzzing in your testing cycles. Not • QA Usage of Fuzzing in Software Development
all fuzzing tools are model-based, but • Regression testing and product comparisons using Fuzzing at test laboratories
fuzzing techniques are always automated • Penetration testing use in IT operations
with almost zero human involvement.
Tests are automatically generated and
As the usage scenarios range from one end to another, so does the profile of the actual
executed, and reports are also typically
users of the tools. Different people look for different aspects in fuzzers. Some users pre-
generated automatically. Most of the
fer random fuzzers, whereas others look for intelligent fuzzing. Other environments
work can be focused on analyzing and fix-
require appliance-based testing solutions, and still other test environments dictate soft-
ing the found issues.
ware-based generators. Fortunately, all of these are readily available today.
Fuzzy Tools
Comparing fuzzing tools is difficult, and compared by running them against a Fuzzing as a security-testing technique
there is no accepted method. The easiest software intentionally planted with seems to have a future. And if you don’t
way might be to enumerate he interface security vulnerabilities. Based on that plan on using it yourself, someone else—
requirements. One toolkit might sup- sample, fuzzer efficiency ranged from quite possibly a hacker—surely will. So
port about 20 or so protocol interfaces 0 percent to 80 percent. Random test- it’s best to fight fire with fire and beat
where another will cover more than 100 ing provided inefficient test results, them at their own game.
protocols. and model-based tests peaked at high- Fuzzing tools are easily accessible as
Testing a Web application requires a er efficiency. The tool with the most free open source tools as well in com-
different set of fuzzers than testing a test cases rarely was the most efficient mercial products. Fuzzing is an efficient
voice over IP (VoIP) method of finding
application. Some FIG. 2: REQUIREMENTS OF FUZZING remotely exploitable
fuzzing frameworks holes in critical sys-
are adept at testing tems, and the return
simple text-based of time and effort
protocols but pro- placed in negative
vide no help for test- testing is immediate.
ing complex struc- Finding just a single
tures such as ASN.1 flaw prior to release
or XML. Other fuzz can save enormous
tests come in costs and time
prepackaged suites resources for internal
with common pro- crisis management,
tocols such as not to mention the
SSL/TLS, HTTP, compromise to a
and UPnP. Still oth- deployed system and
ers might require damage to reputa-
you to build the tests yourself. one. Looking at the number of test tion. No bug can stay hidden if correct
The test direction and physical cases will often lead to selection of a tools are used correctly.
interfaces also can impact the usability tool that has the least intelligence in Still, there is always room for
of some tools, and some test only serv- the test generation. Pleasantly surpris- advancement, and fuzzing research and
er-side implementations in a client- ing, all planted bugs were found by at development are ongoing. ý
server infrastructure, for example. In a least one fuzzer. So in critical environ- REFERENCES
study conducted by Charlie Miller, ments, it might be good to employ a This article was based on Fuzzing for Software Security
which appears in Fuzzing for Software few solutions, rather than entrusting Testing and Quality Assurance (Artech House, 2008)
• Web site: http://www.fuzz-test.com
Security and Quality Assurance all your efforts in a single fuzzing tool • PROTOS project:
(Artech House, 2008), fuzzers were or technology. http://www.ee.oulu.fi/research /ouspg/protos/
Defect Types Detecting defects at the point of injection, rather than later in the development process, also
High-level languages such as C and C++ greatly reduces the cost of remediation and ensures that software quality is not degraded
are commonly used for diverse and far- with excessive maintenance.
reaching types of applications, due to
their inherent flexibility and powerful
•
requirements by detailing sce- opening the file, it is possible
nario-driven threads through to apply static checks to the
the functional requirements. code to uncover common
So, for example, if a user defects.
(the actor) requests to write to Using the same example, a
a file, possible scenario preconditions This form of analysis is able to reveal file open should not follow a previous
may be: data anomalies such as the use of open of the same file without an inter-
a. the file does not exist uninitialized variables, or variables vening file close, as this can lead to
b. the file does exist that have been assigned a value but dangerous race conditions resulting in
Considerations for the interaction are never referenced. abnormal program termination or
include user privileges, filtering the Within the overall process of static data integrity violations.
input and available file space. analysis, there is an initial (main) part Dynamic Analysis involves executing
Static Analysis involves the analysis of analysis that facilitates all further a program with test data and monitor-
of a program without actually execut- analysis. Specifically, it extracts details ing the process. Many aspects of test
T
‘false’, with the other conditions held
HE COST OF DEFECTS constant, produces a change in the
result of the whole decision. A mini-
Motivating the move to defect tracking by general-market software companies is the mum of (n+1) data items is needed to
cost of defects. Recall Barry Boehm’s groundbreaking work in software economics, in achieve full MC/DC.
which he quantified the relative expense to fix a bug at different times in the develop- This extra coverage means that pos-
ment lifecycle. Although his work was based on the waterfall model, and not the now sible errors will be hit and there is a
commonly used iterative development model, the underlying principle remains the greater confidence level in the code
same: that it’s a lot less expensive to correct defects during development, than to cor- when conditions are tested.
rect them after deployment. Unit Testing checks that the outputs
of a unit of code are appropriate to
The figure shows that costs should ideally track as close to the preferred trend analy- the requirements of the unit and that
sis (solid red) line as possible, as opposed to letting this slide over to the less desirable it responds in a known way under all
but often typical (dashed purple) line. In the latter scenario, developers defer all soft- input states. The sensitivity to any gen-
ware application checking to the quality and assurance phase of development which eral fault is enhanced because the out-
results in a much greater cost (black solid line). puts are examined close to the point
of generation, rather than in a com-
plete system where they can be masked
by other activities.
C
Advanced Resource Projects Agency trary code on a system. Dynamic mem-
(DARPA) in November 1988 to deal ory management is generally treated 'MON WORM
with Internet security problems follow- with caution due to the effect a mis-
ing the Morris Worm strike. (See take by a developer may have on the
C’mon Worm) results obtained from a program. Although intended purely as an aca-
Again with reference to the write to From a security viewpoint, resource demic exercise to gauge the size of the
file example, the CERT C Secure depletion and denial of service are the Internet, the effect of the Morris
Coding Standard provides a number underlying rationale for careful check- Worm had repercussions throughout
of guidelines aimed at removing ing of memory management code. the worldwide Internet community,
potential insecurities related to file infecting thousands of machines. Many
char* ptr = (char*)malloc (SIZE);...
input/output. if (abrt) { organizations with systems attached to
Essentially, file handling defects free(ptr); the Internet suffered damaging denial
may allow an attacker to misuse an }
of service attacks. Consequently, soft-
...
application through unchecked or ware vulnerabilities came under the
free(ptr);
unfiltered user input, i.e. the program
microscope of the U.S. government.
assumes that all user input is safe. The Bottom Line is Security
Programs that do not check user input To achieve a secure and reliable soft-
The CERT C Center is located at
can allow unintended direct execution ware system, there are a number of
Carnegie Mellon University’s Software
of commands or SQL well defined steps and
Engineering Institute (SEI). The cen-
statements (known as corresponding V&V tech-
ter was primarily established to deal
buffer overflows, SQL niques that should be
injection or other non-
validated inputs).
One example of this is
where the user is
• applied. The initial focus
in any project should be
on capturing and specify-
ing complete, unambigu-
with Internet security problems in
response to the poor perception of
security and reliability of the Internet.
For a number of years prior to tackling
required to provide a file
The automated ous requirements. programming guidelines and other
name, for the purpose of However, developers security-related activities, the CERT C
storing further input, verification of should also apply diverse Center studied and compiled cases of
which is then created. V&V techniques at all software vulnerabilities. The Secure
However if pathname is design and stages of software devel- Coding Initiative, launched in 2005,
entered together with an opment. In particular, used the database of catalogued vul-
unchecked file name, implemention the automated verifica- nerabilities, built up over a period of
this may lead to a system tion of design and imple- 12-15 years to develop secure coding
file being overwritten. artifacts leads mentation artifacts, practices in C and C++.
The guidelines in namely code, leads to
CERT C are spread across to greater greater confidence in the SEI is also working very closely with
thirteen distinct chapters quality of software. Static sponsors, such as the U.S. Department
and begin by covering confidence in analysis, through the of Homeland Security (DHS) and
language independent enforcement of appropri-
other defense agencies, to correlate
preprocessor directives, software quality. ate programming stan-
vulnerabilities with coding errors. DHS
followed by C language dards, provides a reliable
also sponsor MITRE’s Common
•
specifics: declarations means of removing the
Weakness Enumeration (CWE), which
and initialization through majority of defects prior
classifies software weaknesses that
to error handling and to testing.
lead to vulnerabilities. The CWE now
miscellaneous items. Common coding mis-
Of course, this takes are typically the contains references to CERT C, and
approach to the C lan- source of security vulner- vice-versa, with the intention that
guage is not uncommon, but it is the abilities in today’s software systems. weaknesses may be eliminated by fol-
emphasis upon security issues that sets CERT C can help tackle security-relat- lowing the secure coding standard.
CERT C apart from other coding stan- ed issues for C-language program-
dards. ming. Many real world attacks on soft- The philosophy that underpins the
The CERT C rule MEM31-C states ware systems have been identified as work of the CERT C Center and CWE
that developers should "[f]ree dynam- the result of exploited vulnerabilities is that the majority of vulnerabilities
ically allocated memory exactly once.” which are traceable to preventable can be traced back to a relatively
This rule can be regarded as high- defects. Indeed, relevant CERT C small number of common defects. If
lighting redundant code, which may guidelines are now referenced by these defects can be eradicated using
be confusing to the reader or make MITRE’s Common Weakness Enum- suitable automated V&V techniques
the code more difficult to understand eration CWE) database for newly dis- then as a consequence a much higher
and maintain. covered and disclosed vulnerabilities, level of software security can be
However, double-free vulnerabili- so that developers can explicitly see attained.
ties are viewed by CERT as something the association. Visit cwe.mitre.org to
that may be exploited to execute arbi- find out more. ý
change and provides a logical collabo- gram has gone wrong. This technique enterCustomerInfo(). After the pro-
ration point for enterprise security is a form of fault injection, a common gram receives valid customer informa-
approach taken by security testers and tion, the program processes cus-
Brian Chess is chief scientist and co-founder
security testing tools. The advantage of tomer’s credit card in processCCard()
of security tool maker Fortify Software;
Jacob West manages the company’s Security
the approach is that when the pro- and completes the transaction.
Research Group. gram misbehaves because it has However, if the customer information
received unusual input, the tester has fails to pass basic validation checks,
such as a check to ensure that the In order to identify sources (meth- Listing 2 shows the code from Listing
postal code for the billing address is ods that introduce untrusted data) and 1 modified to include representative
valid, control will not proceed and sinks (methods that untrusted data dynamic taint propagation logic around
processCCard() will never be exer- should never reach) for tainted values, program points that introduce, propa-
cised. Without focused test data, fault we instrument the program to set the gate, or potentially misuse taint. The
injection techniques will not spend as taint-storage values added to the String code added at runtime to permit taint
much time exercising processCCard(), class in cases where values are read propagation is shown in boxes. When a
and so it is more likely to miss bugs in from outside the program and could String is created or updated with untrust-
the program logic found there. be influenced by an attacker. We also ed input, a call to setTaintMarker() is
In many cases this means that fault instrument a variety of security-rele- inserted. When taint is propagated from
injection requires much more time vant methods whose arguments should one string to another, a similar call is
and effort than functional testing to not be controlled by an attacker to used to transfer the taint status to the
achieve the same level of test coverage. check that their sensitive string argu- new string. Finally, before a call to a secu-
Our experience is that many organiza- ments are not tainted. If a security-rel- rity-relevant operation, such as
tions either omit security testing evant method is invoked with a tainted executeQuery(), a call to checkTaint() is
entirely or give it only a fraction of the string, a warning is raised. inserted to check if the argument to the
resources devoted to functional test- To better understand how taint sensitive operation can be controlled by
ing. The result is that many input vali- propagation can be used to identify a an attacker.
dation problems are overlooked. vulnerability, consider the code in
Dynamic taint propagation works by Listing 1, which demonstrates a classic LISTING 2
List getUser(HttpServletRequest request) {
monitoring the target program as it SQL injection vulnerability. In the ...
runs and associating a taint marker
with user-controlled input. The taint FIG. 1: DIDN'T BREAK THE SKIN
marker propagates through the pro-
gram with the input data. If a taint
marker reaches a sensitive function processCCard ( )
before it encounters appropriate input
validation, a vulnerability is reported.
String¬ user = request.getParameter1("user"); that contains a value read from an interfaces or parent classes in an
TaintUtil.setTaintMarker(user, 1);
try {
HTTP request parameter would inheritance hierarchy, in some cases
String sql = "SELECT * FROM users WHERE receive a higher priority than the same we are able to instrument code even
id='" + user + "'"; vulnerability caused by a value read though we have not explicitly written a
TaintUtil.setTaintMarker(sql, from a local properties file. When an rule with it in mind.
user.getTaintMarker());
TaintUtil.checkTaint(sql); error is reported, it includes details
stmt.executeQuery1(sql); about not only the type of vulnerabili- Sources of Inaccuracy
} ty, but also the specific source and sink Here we discuss ways to combat both
...
involved and the line numbers where false positives and false negatives and
}
they are located in the original pro- maximize the accuracy of results pro-
To make dynamic taint propagation gram source code. duced by dynamic taint propagation.
effortless for testers, we modify the Table 1 shows an overview of a vul- In programs where security was
bytecode for the core Java Runtime
Environment (JRE) classes, the pro- TABLE 2: A DIFFERENT BREED
gram’s bytecode and the bytecode of
SQL Injection: A SQL injection issue where external taint reached a database sink.
URL: http://localhost/splc/listMyItems.do Verified: 3
any external libraries the program
employs. We perform the instrumen-
tation at runtime by replacing the Source: Web Input Sink: Database
application server’s class loader with
File: org.apache.coyote.to File: com.order.splc.Item
one designed to rewrite classes target- mcat5.CoyoteReques Service:201
ed for instrumentation as they are tFacade:295 String[]
loaded. Performing instrumentation
Method: org.apache.coyote.to Method: ResultSet
at load-time avoids changes to the pro-
mcat5. java.sql.Statement.
gram’s source code or binary on disk CoyoteRequest. executeQuery(String)
and makes it easy to analyze multiple getParameterValues
programs loaded in the same applica- (String)
tion server. This means the program’s Method Arguments: bean.quantity Method Arguments: select id, account, sku,
build and deployment processes do quantity, price, ccno,
not have to change in order to use Return Value: ' OR 1=1-- description from item
dynamic taint propagation. Rewriting where account = 'gary'
Stack Trace: ... and quantity = '' OR
a class at runtime roughly doubles the 1=1--'
amount of time required for loading HTTP Request: ...
the class, so programs are noticeably Stack Tracer: ...
slower to start. But once a class has
HTTP Request: ...
been loaded, the additional code
required for dynamic taint propaga-
tion adds little overhead to the pro- nerability report for a SQL injection addressed during development, many
gram’s execution time. issue detected with runtime taint false positives are caused by unrecog-
Beyond tracking taint as a binar y propagation. Notice the vulnerability nized input validation because we can-
property of a string, it is often desir- report contains the URL, as well as not automatically determine whether
able to differentiate multiple sources code-level details about the source and an input validation mechanism is suffi-
of taint and track them independently. sink involved in the vulnerability. cient to mitigate a vulnerability. Doing
To address this demand, our taint so would require that we keep track of
tracking mechanism supports taint Writing Rules which specific characters and sub-
flags, which associate information The choice of which classes and meth- strings can make their way through
about sources that introduce taint with ods to instrument has a clear impact the validation logic and relate this
tainted values that they impact. Armed on the effectiveness of our dynamic information to the types of attacks pos-
with detailed information about the taint propagation approach. Instru- sible on each sink. Listing 3 shows the
source of a tainted value when it caus- ment too broadly, and the analysis will SQL injection from Listing 1 mitigated
es a vulnerability to be reported, we produce false positives (also called with whitelist validation that ensures
can report vulnerabilities more accu- false alarms). Instrument too narrow- the untrusted input contains only
rately and include more useful infor- ly, and the analysis will suffer false neg- upper and lower case characters from
mation with the vulnerabilities we atives (miss real vulnerabilities). We the English alphabet. Without knowl-
report. derived the set of classes and methods edge of the constraints Input
When taint reaches a security-sensi- to instrument from the rule set we use Util.alphaOnly() places on the input,
tive sink, we must decide what, if any, for SCA, our static analysis tool. SCA we will report a false positive on the
vulnerability to report. Our taint prop- performs taint propagation on source subsequent call to executeQuery().
agation implementation is capable of code without running it, so converting LISTING 3
fine-grained decisions about the type the rule set for use with dynamic taint List getUser(HttpServletRequest request) {
and priority of error to report depend- propagation was a fast way to create ...
String user = request.getParameter("user");
ing on which source and sink are rules for thousands of packages and if (!InputUtil.alphaOnly(user)) { // ensure user
involved. For example, a SQL query methods. Because rules can refer to matches a-zA-Z
puter science Dongyan Xu and others the PHP interpreter by Core Security tool can explore many more possible
track which bytes in a C program come Technologies (grasp.coresecurity.com) execution paths than would be practical
from user input by reserving a portion that includes taint tracking with char- to exercise during program testing. The
of the program’s address space for acter-level precision. disadvantage of static taint propagation
taint tracking. Every memory location All of the tools mentioned thus far is that less information is available
in the program has an associated entry perform taint propagation at runtime. about the true state of the program, so
in the taint map. As user input information about possible exe-
propagates through the pro- cution paths is necessarily less
•
gram, instrumentation added precise.
to the program updates the
taint map. The implementa- Broader QA Role
tion uses static analysis to elim- Dynamic taint propagation does
inate instrumentation in por-
tions of the code that will
The disadvantage of static taint not rely on fault injection and
does not disrupt the normal
never carry taint. The advan- behavior of the application. For
tage of this low-level and high-
propogation is that less information is this reason, it does not require
ly precise approach is that it any effort beyond standard
can be applied not only to pro-
available about program state. functional testing. By harness-
grams written in C, but also to ing the energy already devoted
programs written in interpret-
ed languages such as PHP
when the interpreter is written
in C.
• to functional testing, dynamic
taint propagation often finds
more input validation bugs than
other security testing approach-
PHP has been the target of es. And because the technique
numerous taint propagation projects, They all associate some shadow state integrates well with existing QA prac-
undoubtedly because PHP has a poor with user input and update that state tices, it seems an effective way for QA
reputation for security and is widely according to the instructions the pro- organizations to contribute to the secu-
used in applications that accept user gram executes. However, taint propaga- rity process. ý
input over a network. PHP does not tion does not have to wait until runtime.
yet have a built-in
klocworkPrint_final.pdf 1 taint propagation
03/03/09 2:52 PM A taint propagation analysis can also be REFERENCES
mechanism, but there’s a version of performed statically. A static analysis 1. http://cwe.mitre.org/documents/vuln-trends
SERIOUS
SOURCECODE
ANALYSIS
Helping software developers create more secure code.
Take the first step towards more secure code – get a free trial of
Klocwork Insight today at www.klocwork.com/freetrialsignup.
Security to delay the project and to remediate the code, or send it out into the market as-is.
Trying to inject security into an application through testing is a fool's errand. The num-
ber of paths through an application is nearly infinite, and you can’t guarantee that all those
Auditing, paths are free of vulnerabilities. It's simply not feasible to identify and test each and every
path for vulnerabilities. Moreover, errors would be difficult to fix considering that the
effort, cost, and time required to fix each bug increases exponentially as the development
One Size process progresses. Most importantly, the bug-finding approach to security fails to address
the root cause of the problem. Security, like quality, must be built into the application.
Building security into an application involves designing and implementing the applica-
tion according to a policy for reducing the risk of security attacks, then verifying that the
Does Not policy is implemented and operating correctly. In other words, security requirements
should be defined, implemented, and verified just like other requirements.
For example, establishing a policy to apply user input validation immediately after the
Fit All Matt Love is a software development manager at Parasoft.
FIG. 1: ONE CODE BRANCH, MULTIPLE INPUTS requirements for the specific applica-
tion under development. Obviously,
this would require considerable inter-
Input action with the internal team members
most familiar with the application.
The security policy should describe
Input Input what types of resources require privi-
leged access, what kind of actions should
be logged, what kind of inputs should be
Switch validated, and other security concerns
If specific to the application. To be sure key
requirements are not overlooked, I rec-
SQL ommend listing all the important assets
Statement that a given application interacts with,
then prioritizing them based on the
importance of protecting each asset.
SQL X Path
Statement Query Applying the Security Policy
SQL Having an effective security policy
Statement defined on paper will not translate to a
secure application unless the security
policy is followed during development.
Static analysis can be used to automati-
input values are received guarantees many regulations, such as Sarbanes- cally verify whether most security policy
that all inputs are cleaned before they Oxley, that require organizations to requirements are actually implemented
are passed down through the infinite demonstrate they have taken "due dili- in the code and identify code that
paths of the code and allowed to wreak gence" in safeguarding application requires rework. Verifying the remaining
havoc (see Figure 1). If this requirement security and information privacy. Yet, security policy requirements might
is defined in the security policy then ver- although the term is mentioned fre- require unit testing, component testing,
ified to be implemented in the code, the quently, it is not often defined. peer code review or other techniques.
team does not need to spend countless A security policy is a specification Using static analysis to automatically
resources finding every bug and testing document that defines how code needs verify the code’s compliance to applica-
every possible user input. tion-specific security policy
One of the best strategies for requirements (for instance, for
building security into the appli-
cation is to define how code
needs to be written to protect it
from attacks, then use static
• authentication, authorization,
logging, and input validation)
requires expressing
requirements as custom static
those
analysis to verify that the policy is An effective security policy on paper analysis rules, then configuring
implemented in the code. This the tool to check those custom
article provides an overview of will not translate to a secure rules. Often, developing such
how this can be accomplished. custom rules is simply a matter of
application unless it's followed. tailoring the static analysis tool’s
Establishing A available security policy rule tem-
Security Policy
•
plates to suit your own policy. For
Writing code without heed for instance, custom SOA security
security then later trying to iden- policy rules can be created from
tify and remove all of the appli- templates such as:
cation’s security vulnerabilities is • Do not import WSDLs
not only resource-intensive, it’s also to be written to protect it from attacks. outside a certain domain
largely ineffective. To have any chance of Security policies typically include cus- • Do not import schemas outside a
exposing all of the security vulnerabili- tom security requirements, privacy certain domain
ties that may be nested throughout the requirements, security coding best prac- Custom Java security policy rules
application, you would need to identify tices, security application design rules, can be created from templates such as:
every single path through the applica- and security testing benchmarks. • Ensure all sensitive method invo-
tion, and then rigorously test each and What do you do if your team does cations are logged
every one. A policy-based approach not already have well-defined security • Allow only certain providers to be
helps alleviate that problem. policy? If the organization has desig- specified for the ''Security.add
Security policies are espoused by nated security experts, they should be Provider()'' method
security experts, such as Open Web writing these requirements. If not, • Keep all access control methods
Application Security Project (OWASP), security consultants could be brought centralized to enforce consistency
and are mandated for compliance with in to help develop appropriate Static analysis can also be used to
check whether code complies with } catch (ISOValidationException e) { a panacea for detecting security vul-
ISOStandardLogger.log(e);
industry-standard security best practices }
nerabilities. It is certainly valuable for
developed for the applicable language quickly exposing vulnerabilities in
and technologies. Many available static XML is no safe haven either. For SOA large code bases without requiring you
analysis tools can check compliance to applications, applying industry-standard to ever write a test case or even run the
such standards “out of the box,” and static analysis rules can expose common application (see Figure 2). However,
with no special configuration. security vulnerabilities that manifest there are some notable shortcomings:
If you are developing in Java, you themselves in XML. For example, static • A complex application has a vir-
would want to perform static analysis analysis could be used to parse the docu- tually infinite number of paths,
to check industry-standard Java securi- ment type definitions (DTDs) that define but data flow analysis can traverse
ty rules such as: XML files and check for recursive entity only a finite number of paths
• Validate an 'HttpServlet Request' declarations that, when parsed, can using a finite set of data. As a
object when extracting data from it quickly explode exponentially to a large result, it finds only a finite num-
• Use JAAS in a single, centralized number of XML elements. If such “XML ber of vulnerabilities.
authentication mechanism bombs” are left undetected, they can con- • It identifies symptoms (where the
• Do not cause deadlocks by calling sume the XML parser and constitute a vulnerability manifests itself)
a synchronized method from a denial of service attack. For instance, stat- rather than root causes (the code
synchronized method ic analysis could be used to identify the that creates the vulnerability).
• Use only strong cr yptographic following DTD that, when processed, Rules-based static analysis exposes
algorithms explodes to a series of 2100 “Bang!” ele- root causes rather than symptoms, and
• Session tokens should expire ments and will cause a denial of service: can reliably target ever y single
• Do not pass mutable objects to instance of that root cause. If you use
<?xml version=”1.0” ?>
'DataOutputStream' in the <!DOCTYPE foobar [
flow analysis, it will probably find you
'writeObject()' method <!ENTITY x0 “Bang!”> a few instances of SQL injection vul-
• Do not set custom security man- <!ENTITY x1 “&x0;&x0;”> nerabilities, but it cannot find them
<!ENTITY x2 “&x1;&x1;”>
agers outside of 'main' method all. However, if you enforce an input
...
For SOA, you would want to check <!ENTITY x99 “&x98;&x98;”> validation rule through rules-based
industry-standard rules such as: <!ENTITY x100 “&x99;&x99;”> static analysis—finding and fixing
• Avoid unbounded schema se- ]> every instance where inputs are not
quence types properly validated—you can guaran-
• Avoid xsd:any, xsd:anyType and Go with the Flow? tee that SQL injection vulnerabilities
xsd:anySimpleType Data flow analysis is often hailed as will not occur.
• Avoid xsd:list types
• Avoid complex types with mixed
content
• Restrict xsd simple types
• Use SSL (HTTPS) in WSDL serv-
ice ports
• Avoid large messages
• Use nonce and timestamp values
in UsernameToken headers
To illustrate how following such
industry-standard rules can prevent secu-
rity vulnerabilities, consider the rule
“Validate an 'HttpServletRequest' object
when extracting data from it.” Following
this rule is important because Http
ServletRequest objects contain user-mod-
ifiable data that, if left unvalidated and
passed to sensitive methods, could allow
serious security attacks such as SQL injec-
tion and cross-site scripting. Because it
allows unvalidated user data to be passed
on to sensitive methods, static analysis
would report a violation of this rule for
the following code:
String name = req.getParameter(“name”);
APRIL 2009
ROCK-HARD SECURITY
Traditional vs.
employees. If an organization wants to
minimize the number of external con-
sultants on an implementation, it must
ensure that the end-users on its imple-
mentation team. It must ensure that the