Data Mining Lab

LABORATORY MANUAL
on
DATA MINING

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

SRI KOTTAM TULASI REDDY MEMORIAL COLLEGE OF ENGINEERING

(Affiliated to JNTU, Hyderabad, Approved by AICTE, Accredited by NBA)
KONDAIR, MAHABOOBNAGAR (Dist), AP - 509125

S.K.T.R.M College off Engineering

Data Mining Lab

1) INTRODUCTION ON WEKA

WEKA (Waikato Environment for Knowledge Analysis) is a popular suite of machine

learning software written in Java, developed at the University of Waikato, New
Zealand.

WEKA is an open source application that is freely available under the GNU general
public license agreement. Originally written in C, the WEKA application has been
completely rewritten in Java and is compatible with almost every computing platform.
It is user friendly with a graphical interface that allows for quick set up and operation.
WEKA operates on the predication that the user data is available as a flat file or
relation. This means that each data object is described by a fixed number of
attributes that usually are of a specific type, normal alpha-numeric or numeric values.
The WEKA application allows novice users a tool to identify hidden information from
database and file systems with simple to use options and visual interfaces.

The WEKA workbench contains a collection of visualization tools and algorithms for
data analysis and predictive modeling, together with graphical user interfaces for
easy access to this functionality.

This original version was primarily designed as a tool for analyzing data from
agricultural domains, but the more recent fully Java-based version (WEKA 3), for
which development started in 1997, is now used in many different application areas,
in particular for educational purposes and research.

2) ADVANTAGES OF WEKA

The obvious advantage of a package like WEKA is that a whole range of data
preparation, feature selection and data mining algorithms are integrated. This means
that only one data format is needed, and trying out and comparing different
approaches becomes really easy. The package also comes with a GUI, which should
make it easier to use.

Portability, since it is fully implemented in the Java programming language and thus
runs on almost any modern computing platform.

A comprehensive collection of data preprocessing and modeling techniques.

Ease of use due to its graphical user interfaces.

WEKA supports several standard data mining tasks, more specifically, data
preprocessing, clustering, classification, regression, visualization, and feature
selection.

All of WEKA's techniques are predicated on the assumption that the data is available
as a single flat file or relation, where each data point is described by a fixed number

S.K.T.R.M College off Engineering

Data Mining Lab

of attributes (normally, numeric or nominal attributes, but some other attribute types
are also supported).

WEKA provides access to SQL databases using Java Database Connectivity and
can process the result returned by a database query.

It is not capable of multi-relational data mining, but there is separate software for
converting a collection of linked database tables into a single table that is suitable for
processing using WEKA. Another important area is sequence modeling.

Attribute Relationship File Format (ARFF) is the text format file used by WEKA to
store data in a database.

The ARFF file contains two sections: the header and the data section. The first line of
the header tells us the relation name.

Then there is the list of the attributes (@attribute...). Each attribute is associated with
a unique name and a type.

The latter describes the kind of data contained in the variable and what values it can
have. The variables types are: numeric, nominal, string and date.

The class attribute is by default the last one of the list. In the header section there
can also be some comment lines, identified with a '%' at the beginning, which can
describe the database content or give the reader information about the author. After
that there is the data itself (@data), each line stores the attribute of a single entry
separated by a comma.

WEKA's main user interface is the Explorer, but essentially the same functionality
can be accessed through the component-based Knowledge Flow interface and from
the command line. There is also the Experimenter, which allows the systematic
comparison of the predictive performance of WEKA's machine learning algorithms on
a collection of datasets.

Launching WEKA
The WEKA GUI Chooser window is used to launch WEKAs graphical environments.
At the bottom of the window are four buttons:
1. Simple CLI. Provides a simple command-line interface that allows direct
execution of WEKA commands for operating systems that do not provide their
own command line Interface.
2. Explorer. An environment for exploring data with WEKA.
3. Experimenter. An environment for performing experiments and conducting.
4. Knowledge Flow. This environment supports essentially the same functions as the
Explorer but with a drag-and-drop interface. One advantage is that it supports
incremental learning.
S.K.T.R.M College off Engineering

Data Mining Lab

If you launch WEKA from a terminal window, some text begins scrolling in the
terminal. Ignore this text unless something goes wrong, in which case it can help in
tracking down the cause. This User Manual focuses on using the Explorer but does
not explain the individual data preprocessing tools and learning algorithms in WEKA.
For more information on the various filters and learning methods in WEKA, see the
book Data Mining (Witten and Frank, 2005).

The WEKA Explorer

Section Tabs
At the very top of the window, just below the title bar, is a row of tabs. When the
Explorer is first started only the first tab is active; the others are greyed out. This is
because it is necessary to open (and potentially pre-process) a data set before
starting to explore the data.
The tabs are as follows:
1. Preprocess. Choose and modify the data being acted on.
2. Classify. Train and test learning schemes that classify or perform regression.
3. Cluster. Learn clusters for the data.
4. Associate. Learn association rules for the data.
5. Select attributes. Select the most relevant attributes in the data.
6. Visualize. View an interactive 2D plot of the data.
Once the tabs are active, clicking on them flicks between different screens, on
which the respective actions can be performed. The bottom area of the window
(including the status box, the log button, and the WEKA bird) stays visible
regardless of which section you are in.

S.K.T.R.M College off Engineering

Data Mining Lab

Classification
Selecting a Classifier
At the top of the classify section is the Classifier box. This box has a text field that
gives the name of the currently selected classifier, and its options. Clicking on the
text box brings up a GenericObjectEditor dialog box, just the same as for filters that
you can use to configure the options of the current classifier. The Choose button
allows you to choose one of the classifiers that are available in WEKA.

Test Options
The result of applying the chosen classifier will be tested according to the options
that are set by clicking in the Test options box. There are four test modes:
1. Use training set. The classifier is evaluated on how well it predicts the class of the
instances it was trained on.
2. Supplied test set. The classifier is evaluated on how well it predicts the class of a
set of instances loaded from a file. Clicking the Set... button brings up a dialog
allowing you to choose the file to test on.
3. Cross-validation. The classifier is evaluated by cross-validation, using the
number of folds that are entered in the Folds text field.
4. Percentage split. The classifier is evaluated on how well it predicts a certain
percentage of the data which is held out for testing. The amount of data held out
depends on the value entered in the % field.
Note: No matter which evaluation method is used, the model that is output is always
the one build from all the training data. Further testing options can be set by clicking
on the More options... button:
1. Output model. The classification model on the full training set is output so that it can
be viewed, visualized, etc. This option is selected by default.
2. Output per-class stats. The precision/recall and true/false statistics for each class
are output. This option is also selected by default.
3. Output entropy evaluation measures. Entropy evaluation measures are included in
the output. This option is not selected by default.
4. Output confusion matrix. The confusion matrix of the classifiers predictions is
included in the output. This option is selected by default.
5. Store predictions for visualization. The classifiers predictions are remembered so
that they can be visualized. This option is selected by default.
6. Output predictions. The predictions on the evaluation data are output. Note that in
the case of a cross-validation the instance numbers do not correspond to the location
in the data!
7. Cost-sensitive evaluation. The errors is evaluated with respect to a cost matrix. The
Set... button allows you to specify the cost matrix used.
S.K.T.R.M College off Engineering

Data Mining Lab

8. Random seed for xval / % Split. This specifies the random seed used when
randomizing the data before it is divided up for evaluation purposes.

The Class Attribute

The classifiers in WEKA are designed to be trained to predict a single class
attribute, which is the target for prediction. Some classifiers can only learn nominal
classes; others can only learn numeric classes (regression problems); still others can
learn both.
By default, the class is taken to be the last attribute in the data. If you want to
train a classifier to predict a different attribute, click on the box below the Test options
box to bring up a drop-down list of attributes to choose from.

Training a Classifier
Once the classifier, test options and class have all been set, the learning process is
started by clicking on the Start button. While the classifier is busy being trained, the
little bird moves around. You can stop the training process at any time by clicking on
the Stop button.
When training is complete, several things happen. The Classifier output area
to the right of the display is filled with text describing the results of training and
testing. A new entry appears in the Result list box. We look at the result list below;
but first we investigate the text that has been output.

The Classifier Output Text

The text in the Classifier output area has scroll bars allowing you to browse the
results. Of course, you can also resize the Explorer window to get a larger display
area. The output is split into several sections:
1. Run information. A list of information giving the learning scheme options,
relation name, instances, attributes and test mode that were involved in the
process.
2.

Classifier model (full training set). A textual representation of the classification

model that was produced on the full training data.

3.

The results of the chosen test mode are broken down thus:

4.

Summary. A list of statistics summarizing how accurately the classifier was able
to predict the true class of the instances under the chosen test mode.

5.

Detailed Accuracy By Class. A more detailed per-class break down of the

classifiers prediction accuracy.

6.

Confusion Matrix. Shows how many instances have been assigned to each
class. Elements show the number of test examples whose actual class
is the row and whose predicted class is the column.

The Result List

After training several classifiers, the result list will contain several entries. LeftS.K.T.R.M College off Engineering

Data Mining Lab

clicking the entries flicks back and forth between the various results that have
been generated. Right-clicking an entry invokes a menu containing these items:
1. View in main window. Shows the output in the main window (just like left-clicking
the entry).
2. View in separate window. Opens a new independent window for viewing the
results.
3. Save result buffer. Brings up a dialog allowing you to save a text file containing
the textual output.
4. Load model. Loads a pre-trained model object from a binary file.
5. Save model. Saves a model object to a binary file. Objects are saved in Java
serialized object form.
6. Re-evaluate model on current test set. Takes the model that has been built and
tests its performance on the data set that has been specified with the Set..
button under the Supplied test set option.
7. Visualize classifier errors. Brings up a visualization window that plots the results
of classification. Correctly classified instances are represented by crosses,
whereas incorrectly classified ones show up as squares.
7. Visualize tree or Visualize graph. Brings up a graphical representation of the
structure of the classifier model, if possible (i.e. for decision trees or Bayesian
networks). The graph visualization option only appears if a Bayesian network
classifier has been built. In the tree visualizer, you can bring up a menu by rightclicking a blank area, pan around by dragging the mouse, and see the training
instances at each node by clicking on it. CTRL-clicking zooms the view out, while
SHIFT-dragging a box zooms the view in. The graph visualizer should be selfexplanatory.
8.

Visualize margin curve. Generates a plot illustrating the prediction margin. The
margin is defined as the difference between the probability predicted for the
actual class and the highest probability predicted for the other classes. For
example, boosting algorithms may achieve better performance on test data by
increasing the margins on the training data.

9.

Visualize threshold curve. Generates a plot illustrating the tradeoffs in

prediction that are obtained by varying the threshold value between classes. For
example, with the default threshold value of 0.5, the predicted probability of
positive must be greater than 0.5 for the instance to be predicted as positive.
The plot can be used to visualize the precision/recall tradeoff, for ROC curve
analysis (true positive rate vs false positive rate), and for other types of curves.

10. Visualize cost curve. Generates a plot that gives an explicit representation of
the expected cost, as described by Drummond and Holte (2000). Options are
greyed out if they do not apply to the specific set of results.

S.K.T.R.M College off Engineering

Data Mining Lab

CREDIT RISK ASSESSMENT

The business of banks is making loans. Assessing the credit worthiness of an

applicants of crucial importance. We have to develop a system to help a loan officer
decide whether the credit of a customer is good or bad. A banks business rules
regarding loans must consider two opposing factors. On the one hand, a bank wants
to make as many loans as possible. Interest on these loans is the banks profit
source. On the other hand, a bank cannot afford to make too many bad loans. To
many bad could leads to the collapse of the bank. The banks loan policy must
involve a compromise not too strict, and not too lenient.

Credit risk is an investor's risk of loss arising from a borrower who does not make
payments as promised. Such an event is called a default. Other terms for credit risk
are default risk and counterparty risk.

Credit risk is most simply defined as the potential that a bank borrower or
counterparty will fail to meet its obligations in accordance with agreed terms.

The goal of credit risk management is to maximise a bank's risk-adjusted rate of

return by maintaining credit risk exposure within acceptable parameters.

Banks need to manage the credit risk inherent in the entire portfolio as well as the
risk in individual credits or transactions.

Banks should also consider the relationships between credit risk and other risks.

The effective management of credit risk is a critical component of a comprehensive

approach to risk management and essential to the long-term success of any banking
organisation.

A good credit assessment means you should be able to qualify, within the limits of
your income, for most loans.

S.K.T.R.M College off Engineering

Data Mining Lab

Lab Experiments
1. List all the categorical (or nominal) attributes and the real-valued
attributes separately.
From the German Credit Assessment Case Study given to us, the following attributes
are found to be applicable for Credit-Risk Assessment:
Total Valid Attributes

Categorical or Nominal
attributes
(which takes True/false,
etc values)

Real valued attributes

1. checking_status
2. duration
3. credit history
4. purpose
5. credit amount
6. savings_status
7. employment duration
8. installment rate
9. personal status
10. debitors
11. residence_since
12. property
14. installment plans
15. housing
16. existing credits
17. job
18. num_dependents
19. telephone
20. foreign worker

1. checking_status
2. credit history
3. purpose
4. savings_status
5. employment
6. personal status
7. debtors
8. property
9. installment plans
10. housing
11. job
12. telephone
13. foreign worker

1. duration
2. credit amount
3. credit amount
4. residence
5. age
6. existing credits
7. num_dependents

S.K.T.R.M College off Engineering

Data Mining Lab

2. What attributes do you think might be crucial in making the credit
assessment? Come up with some simple rules in plain English
using your selected attributes.
According to me the following attributes may be crucial in making the credit risk
assessment.
1. Credit_history
2. Employment
3. Property_magnitude
4. job
5. duration
6. crdit_amount
7. installment
8. existing credit
Based on the above attributes, we can make a decision whether to give credit or not.
checking_status = no checking AND other_payment_plans = none AND
credit_history = critical/other existing credit: good
checking_status = no checking AND existing_credits <= 1 AND
other_payment_plans = none AND purpose = radio/tv: good
checking_status = no checking AND foreign_worker = yes AND
employment = 4<=X<7: good
foreign_worker = no AND personal_status = male single: good
checking_status = no checking AND purpose = used car AND
other_payment_plans = none: good
duration <= 15 AND other_parties = guarantor: good
duration <= 11 AND credit_history = critical/other existing credit: good
checking_status = >=200 AND num_dependents <= 1 AND
property_magnitude = car: good
checking_status = no checking AND property_magnitude = real estate AND
other_payment_plans = none AND
age > 23: good
savings_status = >=1000 AND property_magnitude = real estate: good
savings_status = 500<=X<1000 AND employment = >=7: good
credit_history = no credits/all paid AND housing = rent: bad
savings_status = no known savings AND checking_status = 0<=X<200 AND
existing_credits > 1: good
checking_status = >=200 AND num_dependents <= 1 AND
property_magnitude = life insurance: good
S.K.T.R.M College off Engineering

10

Data Mining Lab

installment_commitment <= 2 AND other_parties = co applicant AND

existing_credits > 1: bad
installment_commitment <= 2 AND credit_history = delayed previously AND
existing_credits > 1 AND
residence_since > 1: good
installment_commitment <= 2 AND credit_history = delayed previously AND
existing_credits <= 1: good
duration > 30 AND savings_status = 100<=X<500: bad
credit_history = all paid AND other_parties = none AND
other_payment_plans = bank: bad
duration > 30 AND savings_status = no known savings AND
num_dependents > 1: good
duration > 30 AND credit_history = delayed previously: bad
duration > 42 AND savings_status = <100 AND
residence_since > 1: bad

S.K.T.R.M College off Engineering

11

Data Mining Lab

3. One type of model that you can create is a Decision Tree - train a
Decision Tree using the complete dataset as the training data.
Report the model obtained after training.
A decision tree is a flow chart like tree structure where each internal node(non-leaf)
denotes a test on the attribute, each branch represents an outcome of the test ,and each
leaf node(terminal node)holds a class label.
Decision trees can be easily converted into classification rules.
e.g. ID3,C4.5 and CART.
J48 pruned tree
1. Using WEKA Tool, we can generate a decision tree by selecting the classify
tab.
2. In classify tab select choose option where a list of different decision trees are
available. From that list select J48.
3. Now under test option ,select training data test option.
4. The resulting window in WEKA is as follows:

5. To generate the decision tree, right click on the result list and select visualize
tree option by which the decision tree will be generated.

S.K.T.R.M College off Engineering

12

Data Mining Lab

6. The obtained decision tree for credit risk assessment is very large to fit on the
screen.

The decision tree above is unclear due to a large number of attributes.

S.K.T.R.M College off Engineering

13

Data Mining Lab

4. Suppose you use your above model trained on the complete dataset,
and classify credit good/bad for each of the examples in the
dataset. What % of examples can you classify correctly? (This is
also called testing on the training set) Why do you think you
cannot get 100 % training accuracy?
In the above model we trained complete dataset and we classified credit
good/bad for each of the examples in the dataset.
For example:
IF purpose=vacation THEN
credit=bad;
ELSE purpose=business THEN
credit=good;
In this way we classified each of the examples in the dataset.
We classified 85.5% of examples correctly and the remaining 14.5% of examples are
incorrectly classified. We cant get 100% training accuracy because out of the 20
attributes, we have some unnecessary attributes which are also been analyzed and
trained. Due to this the accuracy is affected and hence we cant get 100% training
accuracy.

S.K.T.R.M College off Engineering

14

Data Mining Lab

5. Is testing on the training set as you did above a good idea? Why
Why not?

Bad idea, if take all the data into training set. Then how to test the above classification is correctly or
not ?
According to the rules, for the maximum accuracy, we have to take 2/3 of the dataset as
training set and the remaining 1/3 as test set. But here in the above model we have taken
complete dataset as training set which results only 85.5% accuracy.

This is done for the analyzing and training of the unnecessary attributes which does not
make a crucial role in credit risk assessment. And by this complexity is increasing and
finally it leads to the minimum accuracy. If some part of the dataset is used as a training set
and the remaining as test set then it leads to the accurate results and the time for
computation will be less.
This is why, we prefer not to take complete dataset as training set.
UseTraining Set Result for the table GermanCreditData:
Correctly Classified Instances

855

85.5

Incorrectly Classified Instances

145

14.5

Kappa statistic

0.6251

Mean absolute error

0.2312

Root mean squared error

0.34

Relative absolute error

55.0377 %

Root relative squared error

74.2015 %

Total Number of Instances

1000

S.K.T.R.M College off Engineering

15

Data Mining Lab

6. One approach for solving the problem encountered in the previous
question is using cross-validation? Describe what cross-validation
is briefly. Train a
Decision Tree again using cross-validation and
report your results. Does your accuracy increase/decrease? Why?
Cross validation:In k-fold cross-validation, the initial data are randomly portioned into k mutually
exclusive subsets or folds D1, D2, D3, . . . . . ., Dk. Each of approximately equal size.
Training and testing is performed k times. In iteration I, partition Di is reserved as
the test set and the remaining partitions are collectively used to train the model.
That is in the first iteration subsets D2, D3, . . . . . ., Dk collectively serve as the
training set in order to obtain as first model. Which is tested on Di. The second
trained on the subsets D1, D3, . . . . . ., Dk and test on the D2 and so on.

1. Select classify tab and J48 decision tree and in the test option select cross
validation radio button and the number of folds as 10.
2. Number of folds indicates number of partition with the set of attributes.

S.K.T.R.M College off Engineering

16

Data Mining Lab

3. Kappa statistics nearing 1 indicates that there is 100% accuracy and hence all the
errors will be zeroed out, but in reality there is no such training set that gives 100%
accuracy.

Cross Validation Result at folds: 10 for the table GermanCreditData:

Correctly Classified Instances
705
70.5 %
Incorrectly Classified Instances
295
29.5 %
Kappa statistic
0.2467
Mean absolute error
0.3467
Root mean squared error
0.4796
Relative absolute error
82.5233 %
Root relative squared error
104.6565 %
Total Number of Instances
1000
Here there are 1000 instances with 100 instances per partition.

Cross Validation Result at folds: 20 for the table GermanCreditData:

Correctly Classified Instances
Incorrectly Classified Instances
Kappa statistic
Mean absolute error
Root mean squared error
Relative absolute error
Root relative squared error
Total Number of Instances

698
69.8
302
30.2
0.2264
0.3571
0.4883
85.0006 %
106.5538 %
1000

%
%

Cross Validation Result at folds: 50 for the table GermanCreditData:

Correctly Classified Instances
Incorrectly Classified Instances
Kappa statistic
Mean absolute error
Root mean squared error
Relative absolute error
Root relative squared error
Total Number of Instances

709
70.9
291
29.1
0.2538
0.3484
0.4825
82.9304 %
105.2826 %
1000

%
%

Cross Validation Result at folds: 100 for the table GermanCreditData:

Correctly Classified Instances
Incorrectly Classified Instances
Kappa statistic
Mean absolute error
Root mean squared error
Relative absolute error
Root relative squared error
Total Number of Instances
S.K.T.R.M College off Engineering

710
71
290
29
0.2587
0.3444
0.4771
81.959 %
104.1164 %
1000

%
%

17

Data Mining Lab

Percentage split does not allow 100%, it allows only till 99.9%

S.K.T.R.M College off Engineering

18

Data Mining Lab

Percentage Split Result at 50%:
Correctly Classified Instances
Incorrectly Classified Instances
Kappa statistic
Mean absolute error
Root mean squared error

Relative absolute error

Root relative squared error
Total Number of Instances

362
138
0.2725
0.3225
0.4764

72.4
27.6

%
%

76.3523 %
106.4373 %
500

Percentage Split Result at 99.9%:

Correctly Classified Instances
0
0
Incorrectly Classified Instances
1
100
Kappa statistic
0
Mean absolute error
0.6667
Root mean squared error
0.6667
Relative absolute error
221.7054 %
Root relative squared error
221.7054 %
Total Number of Instances
1

S.K.T.R.M College off Engineering

%
%

19

Data Mining Lab

7. Check to see if the data shows a bias against "foreign workers"

(attribute 20), or "personal-status"(attribute 9). One way to do this

(Perhaps rather simple minded) is to remove these attributes from
the dataset and see if the decision tree created in those cases is
significantly different from the full dataset case which you have
already done. To remove an attribute you can use the reprocess tab
in WEKA's GUI Explorer. Did removing these attributes have any
significant effect? Discuss.
This increases in accuracy because the two attributes foreign workers and
personal status are not much important in training and analyzing. By removing this,
the time has been reduced to some extent and then it results in increase in the
accuracy. The decision tree which is created is very large compared to the decision
tree which we have trained now. This is the main difference between these two
decision trees.

After forign worker is removed, the accuracy is increased to 85.9%

S.K.T.R.M College off Engineering

20

Data Mining Lab

If we remove 9th attribute, the accuracy is further increased to 86.6% which shows that these
two attributes are not significant to perform training.

Cross validation after removing 9th attribute.

S.K.T.R.M College off Engineering

21

Data Mining Lab

Percentage split after removing 9th attribute.

After removing the 20th attribute, the cross validation is as above.

S.K.T.R.M College off Engineering

22

Data Mining Lab

After removing 20th attribute, the percentage split is as above.

S.K.T.R.M College off Engineering

23

Data Mining Lab

8. Another question might be, do you really need to input so many
attributes to get good results? Maybe only a few would do. For
example, you could try just having attributes 2, 3, 5, 7, 10, 17 (and
21, the class attribute (naturally)). Try out some combinations.
(You had removed two attributes in problem 7 Remember to reload
the ARFF data file to get all the attributes initially before you start
selecting the ones you want.)
Select attribute 2,3,5,7,10,17,21 and click on invert to remove the remaining attributes.

Here accuracy is decreased.

S.K.T.R.M College off Engineering

24

Data Mining Lab

Select random attributes and then check the accuracy.

After removing the attributes 1,4,6,8,9,11,12,13,14,15,16,18,19 and 20,we select the left
over attributes and visualize them.
S.K.T.R.M College off Engineering

25

Data Mining Lab

After we remove 14 attributes, the accuracy has been decreased to 76.4% hence we can
further try random combination of attributes to increase the accuracy.
Cross
validation

S.K.T.R.M College off Engineering

26

Data Mining Lab

Percentage split

S.K.T.R.M College off Engineering

27

Data Mining Lab

9. Sometimes, the cost of rejecting an applicant who actually has a
good credit
Case 1. might be higher than accepting an applicant who has bad
credit
Case 2. Instead of counting the misclassifications equally in both
cases, give a higher cost to the first case (say cost 5) and
lower cost to the second case. You can do this by using a
cost matrix in WEKA.
Train your Decision Tree again and report the Decision Tree and
cross-validation results. Are they significantly different from
results obtained in problem 6 (using equal cost)?
In the Problem 6, we used equal cost and we trained the decision tree. But here, we
consider two cases with different cost. Let us take cost 5 in case 1 and cost 2 in
case 2.
When we give such costs in both cases and after training the decision tree, we can
observe that almost equal to that of the decision tree obtained in problem 6. Case1
(cost 5) Case2 (cost 5)
Total Cost
3820 1705
Average cost 3.82 1.705
We dont find this cost factor in problem 6. As there we use equal cost. This is the
major difference between the results of problem 6 and problem 9.
The cost matrices we used here:
Case 1: 5 1
1 5
Case 2: 2 1
12

S.K.T.R.M College off Engineering

28

Data Mining Lab

1.Select classify tab.

2. Select More Option from Test Option.

3.Tick on cost sensitive Evaluation and go to set.

S.K.T.R.M College off Engineering

29

Data Mining Lab

4.Set classes as 2.
5.Click on Resize and then well get cost matrix.
6.Then change the 2nd entry in 1st row and 2nd entry in 1st column to 5.0
7.Then confusion matrix will be generated and you can find out the difference
between good and bad attribute.
8.Check accuracy whether its changing or not.

S.K.T.R.M College off Engineering

30

Data Mining Lab

10. Do you think it is a good idea to prefer simple decision trees

instead of having long complex decision trees? How does the
complexity of a Decision Tree relate to the bias of the model?
When we consider long complex decision trees, we will have many unnecessary
attributes in the tree which results in increase of the bias of the model. Because of
this, the accuracy of the model can also effect.
This problem can be reduced by considering simple decision tree. The attributes will
be less and it decreases the bias of the model. Due to this the result will be more
accurate.
So it is a good idea to prefer simple decision trees instead of long complex trees.
1. Open any existing ARFF file e.g labour.arff.
2. In preprocess tab, select ALL to select all the attributes.
3. Go to classify tab and then use traning set with J48 algorithm.

S.K.T.R.M College off Engineering

31

Data Mining Lab

4.

To generate the decision tree, right click on the result list and select visualize tree
option, by which the decision tree will be generated.

5. Right click on J48 algorithm to get Generic Object Editor window

6. In this,make the unpruned option as true .
S.K.T.R.M College off Engineering

32

Data Mining Lab

7. Then press OK and then start. we find the tree will become more complex if not
pruned.

Visualize tree

8. The tree has become more complex.

S.K.T.R.M College off Engineering

33

Data Mining Lab

S.K.T.R.M College off Engineering

34

Data Mining Lab

11. You can make your Decision Trees simpler by pruning the node
s. One approach is to use Reduced Error Pruning - Explain this
idea briefly. Try reduced error pruning for training your Decision
Trees using cross-validation (you can do this in WEKA) and report
the Decision Tree you obtain? Also, report your accuracy using
the pruned model. Does your accuracy increase?
Reduced-error pruning:The idea of using a separate pruning set for pruningwhich is applicable to decision trees
as well as rule setsis called reduced-error pruning. The variant described previously
prunes a rule immediately after it has been grown and is called incremental reduced-error
pruning.
Another possibility is to build a full, unpruned rule set first, pruning it afterwards by
discarding individual tests.
However, this method is much slower. Of course, there are many different ways to assess
the worth of a rule based on the pruning set. A simple measure is to consider how well the
rule would do at discriminating the predicted class from other classes if it were the only rule
in the theory, operating under the closed world assumption.
If it gets p instances right out of the t instances that it covers, and there are P instances of
this class out of a total T of instances altogether, then it gets positive instances right. The
instances that it does not cover include N - n negative ones, where n = t p is the number
of negative instances that the rule covers and N = T - P is the total number of negative
instances.
Thus the rule has an overall success ratio of [p +(N - n)] T , and this quantity, evaluated on
the test set, has been used to evaluate the success of a rule when using reduced-error
pruning.
1. Right click on J48 algorithm to get Generic Object Editor window
2. In this,make reduced error pruning option as true and also the unpruned option as
true .
3. Then press OK and then start.

S.K.T.R.M College off Engineering

35

Data Mining Lab

4. We find that the accuracy has been increased by selecting the reduced error pruning
option.
S.K.T.R.M College off Engineering

36

Data Mining Lab

12. (Extra Credit): How can you convert a Decision Trees into "if-thenelse rules".
Make up your own small Decision Tree consisting of 2-3 levels
and convert it into a set of rules. There also exist different
classifiers that output the model in the form of rules - one such
classifier in WEKA is rules. PART, train this model and report
the set of rules obtained. Sometimes just one attribute can be
good enough in making the decision, yes, just one! Can you
predict what attribute that might be in this dataset? OneR
classifier uses a single attribute to make decisions (it chooses the
attribute based on minimum error). Report the rule obtained by
training a one R classifier. Rank the performance of j48, PART
and oneR.
In WEKA, rules.PART is one of the classifier which converts the decision trees into IFTHEN-ELSE rules.
Converting Decision trees into IF-THEN-ELSE rules using rules.PART classifier:PART decision list
outlook = overcast: yes (4.0)
windy = TRUE: no (4.0/1.0)
outlook = sunny: no (3.0/1.0)
: yes (3.0)
Number of Rules : 4
Yes, sometimes just one attribute can be good enough in making the decision.
In this dataset (Weather), Single attribute for making the decision is outlook
outlook:
sunny -> no
overcast -> yes
rainy -> yes
(10/14 instances correct)
With respect to the time, the oneR classifier has higher ranking and J48 is in 2nd place
and PART gets 3rd place.
J48 PART oneR
TIME (sec) 0.12 0.14 0.04
RANK II III I
But if you consider the accuracy, The J48 classifier has higher ranking, PART gets
second place and oneR
gets lst place
J48 PART oneR
ACCURACY (%) 70.5 70.2% 66.8%

1.Open existing file as weather.nomial.arff

2.Select All.
3.Go to classify.
4.Start.

S.K.T.R.M College off Engineering

37

Data Mining Lab

Here the accuracy is 100%

S.K.T.R.M College off Engineering

38

Data Mining Lab

The tree is something like if-then-else rule

If outlook=overcast then
play=yes
If outlook=sunny and humidity=high then
play = no
else
play = yes
If outlook=rainy and windy=true then
play = no
else
play = yes

S.K.T.R.M College off Engineering

39

Data Mining Lab

To click out the rules

1. Go to choose then click on Rule then select PART.

2. Click on Save and start.
3. Similarly for oneR algorithm.

S.K.T.R.M College off Engineering

40

Data Mining Lab

If outlook = overcast then

play=yes
If outlook = sunny and humidity= high then
play=no
If outlook = sunny and humidity= low then
play=yes

S.K.T.R.M College off Engineering

41