Proceedings of the 8th WSEAS International Conference on APPLIED COMPUTER SCIENCE (ACS'08)

Mining Sequential Patterns by PrefixSpan algorithm

with approximation
ANKHBAYAR YUKHUU* SANSARBOLD GARAMRAGCHAA
and HWANG YOUNG SUP**
Department of Computer Science,
Sun Moon University,
100 Kalsan-ri, Tangjeongmyeon, Asan ; Chugnam, 336-708
SOUTH KOREA

Abstract- We want to find sequential patterns in a long continues noisy DNA sequence. Sequential pattern mining, which
discovers frequent subsequences as patterns in a sequence database, is an important data mining problem with broad
applications, including the analysis of customer purchase patterns or Web access patterns and analysis of DNA
sequences, and so on. We investigated sequential pattern mining algorithms for long continues DNA sequences. Most
previously proposed mining algorithms follow the exact matching with a sequential pattern definition. They are not able
to work in noisy environments and inaccurate data in practice. We investigated approximate matching method to deal
with those cases. In this paper, we develop and apply Pattern–Growth PrefixSpan algorithm to find most repeated
patterns, for example, motifs in DNA sequence. Our algorithm gains its efficiency by using pattern growth and
approximation methodologies. The algorithm is based on the observation that all occurrences of a frequent pattern can be
classified into groups, which we call approximated pattern. We developed algorithms to quickly find out all relative
frequents by a pattern growth method and to determine approximated patterns from those frequents. Our experimental
studies demonstrate that our algorithm is efficient in mining repeated approximate sequential patterns that would have
been missed by existing methods.

Key-Words: - sequential pattern mining, searching sequential pattern, pattern growth method, PrefixSpan algorithm with
approximation, long DNA sequence mining, searching subsequences.

1 Introduction Prefixspan extremely run faster than the a priori-based

The sequential pattern mining problem was first
introduced by Agrawal and Srikant in [1]: Given a set of
sequences, where each sequence consists of a list of
elements and each element consists of a set of items, and
given a user-specified minimum support(user specified
minimum length frequent) threshold, sequential pattern
mining is to find all frequent subsequences, i.e., the
subsequences whose occurrence frequency in the set of
sequences is no less than the minimum support.
Sequential pattern mining is an important data mining
problem with broad applications, including the analyses
of customer purchase behavior, Web access patterns,
scientific experiments, disease treatments, natural
disasters, DNA sequences, and so on[2]. There are too
many previously proposed pattern mining methods based
on a priori-like methods: typical a priori-like method such
as GSP (Generalized Sequence Pattern) adopts a
multiple-pass, candidate generation, test approach in
sequential pattern mining [2][3][4][5]. Also Freespan and
GSP algorithm [3][4][5].
Almost existing mining algorithms follow the exact
matching with a sequential pattern definition. They are
not able to work in noisy environments and inaccurate
data in practice. We studied approximate matching
method to deal with such as DNA sequence. The
development of efficient and scalable algorithms for
mining approximate sequential patterns is a challenging
and practically useful direction to pursue. It has been
shown that the capacity to accommodate approximation
in the mining process has become critical due to inherent
noise and imprecision in data or gene mutations in
genomic DNA sequences [8]. The notion of approximate
sequential pattern has been proposed in [8], in which an
algorithm called ApproxMap is designed to mine
consensus patterns. While mining consensus patterns
provides one way to produce compact mining result
under general distance measures, it remains a challenge
how to efficiently mine the complete set of approximate

ISSN: 1790-5109 159 ISBN: 978-960-474-028-4

 Proceedings of the 8th WSEAS International Conference on APPLIED COMPUTER SCIENCE (ACS'08)
sequential patterns under some distance measure which is
stricter yet equally useful in many cases.
Classical exploratory data analysis methods used in
statistics and many of the earlier KDD methods tend to
focus only on basic data types, such as interval or
categorical data, as the unit of analysis. However, some
information cannot be interpreted unless the data is
treated as a unit, leading to complex data types. For
example, the research in DNA sequences involves
interpreting huge databases of amino acid sequences. The
interpretation couldn’t be obtained if the DNA sequences
were analyzed as multiple categorical variables. The
interpretation requires a view of the data at the sequence
-level. DNA sequence research has developed many
methods to interpret long sequences of alphabets.
The purpose of this study is efficiently mining long
sequential patterns in noisy environments. In this paper,
we develop and apply Pattern–Growth PrefixSpan
algorithm to find most repeated patterns, motifs in DNA
sequence. We establish a more general model. The
sequences in its support set may have different patterns of
substitutions; they can in fact be classified into groups,
which we call approximated patterns. Each approximated
pattern is a set of sequences sharing a unified pattern
representation together with its support.
The idea is that by pattern growth algorithm the mined
sets of the patterns discover their related approximated
patterns, and we are able to classify them. These initial
approximated patterns will then be iteratively assembled
into longer approximated patterns in a local search
fashion, until no longer ones can be found. In the second
stage, generalize the longest or most repeated patterns
from grouped based on their constituting sequences to
form a support set so that the frequent approximate
patterns would be identified. Instead of mining only the
patterns repeating within a sliding window of fixed sizes,
our algorithm is able to mine all globally repeating
approximate patterns.
Traditionally in data mining, sequential patterns have
been defined as a subsequence that appears frequently in
a sequence database. Such definition leads to a support
model based sequential pattern mining methods [3].
Currently as far as we know, the only methods available
for analyzing sequences of sets are based on such a
support model.
Although the support model based sequential pattern
mining has been extensively studied and many methods
have been proposed, there are some inherent obstacles
within conventional model. These methods meet inherent
difficulties in mining databases with long sequences and
noise. These limitations are discussed with our algorithm.
The most common sequential pattern mining method is
Mining Patterns by Pattern growth [3]: divide sequential
patterns to be mined based on the subsequences obtained
ISSN: 1790-5109
so far and project the sequence database based on the
partition of such patterns. The general idea is outlined as
follows: Instead of repeatedly scanning the entire
database, generate and test large sets of candidate
sequences. One can recursively project a sequence
database into a set of smaller databases associated with
the set of patterns mined so far and, then, mine locally
frequent patterns in each projected database.
2 Term definition
Definition 1- If sequence X is corresponded to (x1, x2, …,
xi) where xi is an element of the sequence. The value of
instances of items in a sequence is length (l) of the
sequence.
Definition 2 - If X sequence is subsequence of another
sequence Y=<y1,y2, ..., ym>, if there exist integers
1<i1<i2<…<in<m such that x1=yi1, x2=yi2, …, xn=yin.
Definition 3 - An X sequence and a user-specified error
threshold minimum support θ, a sequence is said to be
frequent if it is contained in most times with θ frequents
in the sequence. A sequential pattern is a maximal
sequence that is frequent.
In the case of DNA sequence: The element of sequence
Xi=A,C,G,T. If X sequence includes a P sequence at least
θ times, a sequence P is called a sequential pattern and
integer θ is called user-specified repeat threshold.
In our case we studied to pattern recognition algorithm to
find a DNA pattern sequence in long sequence with noise
by pattern growth algorithm with error recognition by
user specified error threshold.
3 Related Algorithms
In this section, we first outline a projection-based
sequential pattern mining method, called FreeSpan, and
then introduce an improved PrefixSpan. Both methods
generate projected databases, but they differ at the criteria
of database projection: FreeSpan creates projected
databases based on the current set of frequent patterns
without a particular ordering (i.e., growth direction),
whereas PrefixSpan projects databases by growing
frequent prefixes. PrefixSpan is substantially faster than
FreeSpan in most sequence databases.
3.1 Freespan
FreeSpan is based on the following property: If an itemset
X is infrequent, any sequence whose projected itemset is a
superset of X cannot be a sequential pattern. FreeSpan
mines sequential patterns by partitioning the search space
and projecting the sequence subdatabases recursively
based on the projected itemsets.
Given the database S and min_support θ1, FreeSpan first
scans S, collects the support for each item, and finds the
set of frequent items.
Frequent items are listed in support descending order.
160 ISBN: 978-960-474-028-4
 Proceedings of the 8th WSEAS International Conference on APPLIED COMPUTER SCIENCE (ACS'08)
f_list = a: 4, b: 4, c: 4, d: 3, e: 3, f: 3
Table 1. A sequence database
id_number Sequence
110 (a(abc)(ac)d(cf))
120 ((ad)c(bc)(ae))
130 ((ef)(ab)(df)cb)
140 (eg(af)cbc)
Algortihm1: Freespan algorithm process for mining each
projected database using the sequence database in Table
1 is detailed as follows:
I. Mining sequential patterns containing only item a.
By scanning sequence database once, the only two
sequential patterns containing only item.
II. Mining sequential patterns containing item b but no
item after b in the f_list. This can be achieved by
constructing the {b}- projected database. For a
sequence α containing item b in S sequence, a
subsequence α' is derived by removing from α all
items after b in f_list. α′ is inserted into {b}-
projected database. Thus {b}-projected database
contains four sequences: )a(ab)a), (aba) , ((ab)a).
By scanning the data project once more, all
sequential patterns containing item b but no item
after b in f_list are found. They are (b),( ab)
III. Mining other subsets of sequential patterns. Other
subsets of sequential patterns can be mined
similarly on their corresponding projected
databases. This mining process proceeds
recursively, which derives the complete set of
sequential patterns.
3.2 PrefixSpan
PrefixSpan algorithm major idea is: any frequent
subsequences can always be found by growing frequent
prefixes. The PrefixSpan algorithm steps are:
I. Find length -1 sequential patterns. Scan S once to
find all frequent items in sequences. Each of those
frequent items is a length -1 sequential pattern.
They are <a>:4 , <b>:4, <c>:4, <d>:3, < e>:3
and <f>:3, where <pattern> : count represents the
pattern and its associated support count.
II. Divide search spaces. The complete set of
sequential patterns can be partitioned into the
following six subsets according to the six prefixes:
(1) Ones having prefix a; Ones having prefix b;…
(6) the ones having prefix <f>;
III. Find subsets of sequential patterns. The subset of
sequential patterns can be mined by constructing
corresponding projected databases and mine each
recursively.
PrefixSpan algorithm is created for mining in projected
databases. In this study our database is a long continuous
sequence. So we used PrefixSpan algorithm and to mine a
continuous sequence database. Significant issue of
ISSN: 1790-5109
PrefixSpan algorithm’s main attribute is that PrefixSpan
only grows longer sequential patterns from the shorter
frequent this was our useful source of idea.
Algorithm of PrefixSpan as follows:
Input: A sequence database S, and the minimum
support threshold θ
Output: The complete set of sequential patterns
Method: Call PrefixSpan (<>, 0, S)
Subroutine PrefixSpan (α, l, S|α )
Parameters: α sequential pattern; l the length of α,
the S|α -projected database, if α≠<>; otherwise
the sequence database S.
4 PrefixSpan algorithm with
approximation
One particularly important problem in the area of
sequential pattern mining is the problem of discovering
all subsequences that appear on a given sequence
database and have minimum support threshold. The
difficulty is in figuring out what sequences to try and then
efficiently finding out which of those are frequent [2].
Pattern-growth methods are a more recent approach to
deal with sequential pattern mining problems. The key
idea is to avoid the candidate generation step altogether,
and to focus the search on a restricted portion of the initial
database.
PrefixSpan is the most promising of the pattern-growth
methods and is based on recursively constructing the
patterns, and simultaneously, restricting the search to
projected databases[3]. A database is the set of
subsequences, which are suffixes of the sequences that
have prefix. In each step, the algorithm looks for the
frequent sequences with prefix a, in the correspondent
projected database.
In this way, the search space is reduced at each step,
allowing for better performances in the presence of small
support thresholds. In the presence of gap constraints, the
algorithm has to be adapted, and this claim is no longer
valid. However, as been shown, the new version of
PrefixSpan remains faster than other methods.
4.1 Approximate matching
A fundamental problem of the conventional methods is
that the exact match on patterns does not take into
account the noise in the data. This causes two potential
problems. In real data, long patterns tend to be noisy and
may not meet the support level with exact matching. Even
with moderate noise, a frequent long pattern can be
mislabelled as an infrequent pattern [8]. Not much work
has been done on approximate pattern mining.
We can extend the conventional sequential pattern
mining model to get an approximate sequential pattern
mining model. Given a minimum threshold distance σ,
161 ISBN: 978-960-474-028-4
 Proceedings of the 8th WSEAS International Conference on APPLIED COMPUTER SCIENCE (ACS'08)

the patterns generated with approximate matching *pattern), the second is searching patterns
depend on the number of string matching transition σ. SearchPrefix(sequence, pattern).
Instead of generating the possible sequences, considering sequential patterns from the shorter frequent ones. And
possible errors, we verify if each sequence element classify them by lengths (length) and by repeats.
performs a valid transition in the automaton, beginning Algorithm 2:
on the initial state. Whenever an element does not
CreateseqPref(char *pattern)
correspond to a valid transition, the algorithm tries to
// Creating pattern for searching function by
replace it (which corresponds to apply a Replacement), pattern growth approach
and if this fail, then tries to ignore (which corresponds to { ...
a Deletion) and finally by trying to introduce a valid return pattern;
transition (which corresponds to an Insertion). Those }
insertions and replacements try correct element by every SearchPrefix(sequence, pattern)
// Searching pattern in sequence for find out
possible element {frequent length is constant}. The repeat of pattern
operation does not depend on the size of the number of { ...
different elements in the database, but only on the DNA Boolean approxAcc(Sequence, int i, pattern,
nucleotides of sequences. int e, int Th);
In our algorithm we use edit operations in approximate return repeat;
}
string matching, 3 transitions as follows:
Boolean approxAcc(Sequence, int i, pattern,
Insertion- Ins(x, i) - adds symbol x to the position i; int e, int Th)
Deletion- Del(x. i) - deletes the symbol x from the // Approximate (by Th – error threshold) in
sequence at position i; searching process
{
Replacement – Repl(x, y, i) – substitutes the symbol x,
if (e > Th)
that occurs at position i, by symbol y return false; // Exceeds maximal error
Example 3: If we are searching ATAT– motif, we will if (I < |s|)
find all occurrences with approximate string matching return AccTrans(sequence,s[i], i,
with pattern,e,Th);
Insertion ATAAT, // Test the achievement of final state
Deletion ATT, and else if (pattern in Sequence)
return true; // Try to achieve a final state
Replacement ATGT … etc
else
Unfortunately, the above model may suffer from the return accTrans(sequence, ( ), i, pattern,
following two problems. First, the mining may find many Th, e);
short and probably trivial patterns. Short patterns tend to }
be easier to get similarity counts from the sequences than Boolean accTrans(Sequence,S [i], int i ,
long patterns. Thus, short patterns may overwhelm the pattern, int Th, e)
// Function for checking transitions by every
results. element
Second, the complete set of approximate sequential {
patterns may be larger than that of exact sequential int dif = max(s[i],[a]-|a in s[i]|)
patterns and thus difficult to understand. By if ( |s[i]|= |a in s[i]|)
approximation, a user may want to get and understand the ok = approxAcc(s,i+1,pattern,Th.e); // found
the transition
general trend and ignore the noise. However, a native else
output of the complete set of approximate patterns in the ok = approxAcc(s,i+1,pattern,e+dif,Th);
above model may generate many (trivial) patterns and //Check edits operations/replacement
thus obstruct the mining of information. if (ok)
4.2 PrefixSpan algorithm with approximate return true;
// Element is not valid delete;
matching ok = approxAcc(s,i+1,pattern,e+s[i] ,Th);
We have used PrefixSpan algorithm to find exact patterns, if (ok)
the approximate matching can identify which return true; // If deletion fails then check
approximate pattern is related. insertion;
We created Algorithm 2, PrefixSpan, approximate ok = approxAcc(s,i+1,pattern,e+[a] ,Th);
if (ok)
matching algorithms which to mine sequential patterns.
return true;
Input data is a long continuous DNA sequence. return false ; }
Our algorithm is divided into two main sections, the first
creating patterns CreateseqPref(char

ISSN: 1790-5109 162 ISBN: 978-960-474-028-4

 Proceedings of the 8th WSEAS International Conference on APPLIED COMPUTER SCIENCE (ACS'08)

First, creating section defines which frequent is next Example 4:

searching process. Creating pattern step is creating longer AAGTATTGCGCATACTTCTTATGGGAGATCCAAATTCC
Second, searching pattern section is finding subsequence CAGTACATTAGTTGACTCTCCTTCCTTAATGTTTTCCT
from main sequence with approximate matching CTGTACGATTATTTATGTTTAAAC…
algorithm, the section returns a pattern repeat. The rule is A text file consist A, G, C, T nucleotides. Algorithm
comparing two sequences with approximation outputs were most repeated patterns and their repeats. We
transitions. have used 27,126 length DNA sequence and user defined
When searching short patterns (length=1; error threshold. It means that one transition is possible.
Table 2 illustrates experimental results of two algorithms.
length=2; lentgh=3) the algorithm may meet
When a pattern length was three {length=3}, output
collision. Furthermore, if pattern length is too long,
pattern of the proposed algorithm was repeated 2 times
pattern mining result would be reduced. However, we are
more than PrefixSpan algorithm’s and last nucleotides
using user defined coefficient minimum length,
were different {TTT, TTA}. Two algorithms are
min_length and maximum length, max_length to
observed that they can find motif can be dissimilar at
solve those problems.
length to length. Different patterns between the outputs
CreateseqPref(char *pattern) is creating Table 2: Comparison of the results achieved without and
patterns to search sequence by pattern growth approach, with approximation.
and add to all possibilities of one element for pattern.
SearchPrefix(sequence, pattern) function Pattern
1 2 3 4 5
length
returns how many times pattern is repeated in sequence. PrefixSpan
Boolean approxAcc(Sequence, int i, Pattern
T TT TTT TTTT TTTTT
pattern, int e, int Th) is checking error exceed Pattern with
T TT TTA TTAT TTATT
Approximation
and final states, Boolean accTrans(Sequence,S PrefixSpan
[i], int i , pattern, int Th, e) is finding Repeat
9,288 2,646 936 270 90
and recognizing transitions. Th is user defined error Repeat with
9,288 2,646 2,015 1,350 520
threshold and e is a temporary error. Approximation
Whenever an element doesn’t correspond to valid
transition, the algorithm tries to replace it (Replacement), showed hidden unknown information to us. Because,
and if this fail , then tries to ignore (Deletion) and finally many patterns can include some transitions in continues
by trying to introduce a valid transitions (Insertion) long noisy DNA sequence.
instead of trying every possible elements(A,C,G,T). Figure 1 compares of the repeats of PrefixSpan
Transitions are limited by user defined threshold Th can algorithm and approximated PrefixSpan algorithm. The
be 1 or 2, because Th increased by 1 searching process repeat desperately rose with approximation. One and two
time would be increase at least 3 times. length pattern is not approximated with our algorithm.
The 3 length repeats has increased 936 to 2,015, when
5. Experimental Results length was 4 repeats were 270 and 1,350 and when
finding 5 length PrefixSpan algorithm repeat was 90, our
Our goal is to validate our claim that approximations
algorithm showed 520 times repeated pattern /ATAGT/.
allow the discovery of unknown information, keeping the
The repeats of our algorithm were always more than other
mining process centered on the user. In order to do it, we
algorithm and showed dissimilar patterns during the
compared general Prefixspan algorithm and the
experiment. It can be observed that PrefixSpan
Prefixspan algorithm with an approximation approach
with approximation algorithm can effectively uncover the
when finding a motif in a DNA sequence.
hidden trend in the DNA sequence. That comparison
This PrefixSpan pattern growth method has been used in
shows that our algorithm can discover more patterns,
most studies in a sequential pattern mining, and it
trends than PrefixSpan algorithm. In result, we are able to
generates datasets that mimic real-world transactions,
find approximately the motifs, most repeated patterns
where customers tend to make a sequence of transactions
from a long noisy DNA sequence.
involving a set of items. Sequence and transaction sizes
In summary our experiment shows that the PrefixSpan
are clustered around a mean and some of them may have
with approximation algorithm is more efficient than non
larger sizes.
approximated PrefixSpan in finding a motif in a long
Input data was a long continuous DNA sequence text file.
repeated pattern.
The first 100 element of sequence is as follows:

ISSN: 1790-5109 163 ISBN: 978-960-474-028-4

 Proceedings of the 8th WSEAS International Conference on APPLIED COMPUTER SCIENCE (ACS'08)
3000
2500
2000
repeats
1500
1000
500
0
2 3
pattern length
PrefixSpan Repeat Repeat with Approximated )
Figure 1: Comparison of the results achieved with and
without approximation. (27,126 length DNA sequence)
6. Conclusions
We proposed the definition of closed frequent
approximate sequential patterns to solve the problem of
mining the complete set of frequent approximate
sequential pattern mining. Our study shows that
PrefixSpan algorithm with approximation mines the
complete set of patterns and it is more efficient than
PrefixSpan algorithm.
Our algorithm is based on the notion of classifying a
pattern’s support set into approximated patterns. We
combine approximation approach mining and iterative
pattern growth. We adopt a local search optimization
technique to completeness of the mining result. Our
performance study shows that our algorithm is able to
mine out globally repeating approximate patterns in
biological genomic DNA data with great efficiency
(when length=4, 5 times better etc.). It is important to
explore how to further develop such a pattern-growth
based sequential mining methodology for effectively
mining DNA databases.
Acknowledgments
This work was supported by the Korea Research
Foundation. Grant funded by the Korean Government
(MOEHRD), (KRF-2006-311-D00833).
References
[1] R. Agrawal and R. Srikant. “Fast algorithms for
mining association rules,” In Proc. 1994 Int. Conf.
Very Large Data, Bases (VLDB’94), pages 487–499,
Santiago, Chile, Sept. 1994.
[2] Antunes C and Oliveira A.L: “Generalization of
Pattern-Growth Methods for Sequential Pattern
ISSN: 1790-5109
Mining with Gap Constraints,” in Int'l Conf Machine
Learning and Data Mining, (2003) 239-251
[3] J. Pei, J. Han, B. Mortazavi-Asl, J. Wang, H. Pinto,
Q. Chen, U. Dayal, Mei-Chun Hsu “Mining
Sequential Patterns by Pattern-Growth: The
PrefixSpan Approach,” IEEE Transactions on
knowledge and engineering , vol. 16, No. 10, october
2004
[4] J. Pei, J. Han, B. Mortazavi-Asl, H. Pinto, Q. Chen,
U. Dayal, and M.-C. Hsu, “PrefixSpan: Mining
Sequential Patterns Efficiently by Prefix-Projected
Pattern Growth,” Proc. 2001 Int. Conf. on Data
4 5
Engineering (ICDE'01), Heidelberg, Germany, April
2001.
[5] Odysseas Efremides, George Ivanov “A Study for
Data Pool Dynamic Workload Approaches of a String
Matching Algorithm over a Heterogeneous Cluster ”
Proceedings of the WSEAS Transactions on
Computers, Issue 7, Volume 5, July 2007
(pp-1466-1474)
[6] M. J. Zaki, “SPADE: An Efficient Algorithm for
Mining Frequent Sequences,” in Machine Learning
Journal, special issue on Unsupervised Learning
(Doug Fisher, ed.), pages 31-60, Vol. 42 Nos. 1/2,
Jan/Feb 2001.
[7] J. Han, J. Pei, and Y. Yin. “Mining frequent patterns
without candidate generation,” In Proc. 2000
ACM-SIGMOD Int. Conf. Management of Data
(SIGMOD’00), pages 1–12, Dallas, TX, May 2000.
[8] H.C. Kum, J. Pei, W. Wang, and D. Duncan.
Approx-MAP : “Approximate Mining of Consensus
Sequential Patterns,” Technical Report TR02-031,
UNC-CH, 2002.
[9] Antunes, C. and Oliveira, A.L., “Sequential Pattern
Mining with Approximated Constraints,” Int. Conf
Applied Computing, IADIS (2004) 131-138
[10] Rachel Akimana, Oliver Markovitch and Yves
Roggeman “Grid’s Confidential Outsourcing of
String Matching ” Proceedings of the 6th WSEAS Int.
Conf. on Software Engineering, Parallel and
Distributed Systems, Corfu Island, Greece, February
16-19, 2007 (pp-63-68)
[11] M Yazid M Saman, M Nordin A Rahman Aziz
Ahmad and A Osman M Tap “A Minimum Cost
Process in Searching for a Set of Similar DNA
Sequences ” Proceedings of the 5th WSEAS
International Conference on Telecommunications
and Informatics, Istanbul, Turkey, May 27-29, 2006
(pp348-353)
[12] M. Yazid M. Saman, M. Nordin A. Rahman, Aziz
Ahmad, A. Osman M. Tap “Incremental Clustering as
a Tool for CRM” Proceedings of the WSEAS
Transactions on Computers, Issue7, Volume 5, July
2007 (pp-1525-1533)
164 ISBN: 978-960-474-028-4