Dichotomous Classification using Transitive Learning

Christopher Brown
Department of Linguistics
University of Texas at Austin
1 University Station B5100
Austin, TX 78712-0198 USA
chbrown@mail.utexas.edu

Abstract scribe various complications of and solutions to

The classifier implementation outlined in
this document is intended to take two cor-
pora that markedly differ from each other
in some general way, to learn the parame-
ters that best distinguish the two corpora,
and then be able to apply those parameters
to a new set of data efficiently.
In this paper I will implement a classi-
fier to distinguish between formal and col-
loquial language use, trained upon col-
loquial data from blogs, Simple English
Wikipedia, and children’s literature, and
formal data from the N.Y. Times, WSJ,
and legal/political/academic texts. The in-
tent of the classifier is then to classify ad-
missions essays on a scale of formality,
which will presumably serve as some in-
dicator of academic success/potential.
1 Introduction
Usually, we’d have a little bit of labeled data,
with very stark, clear, decisive labels, and a lot
of mushy, who-knows-what unlabeled data, that is
pretty much the same kind of document as my la-
beled data.
In my case, I have a ton of very thinly labeled
data that isn’t much like what I want to end up
classifying. The labeling is kind of fuzzy – i.e.
it’s very noisy – but my unlabeled data is benefi-
cial because it is very similar to the data I want to
classify.
It is difficult to label certain lexical traits, such
as linguistic register, in a general low-level way.
Spelling correction is sub-optimal in many of its
modern implementations. While some text editing
programs allow personal dictionaries, none that I
know of track personal mistake tendencies. Two
treatments of spelling correction algorithms, (Ju-
rafsky and Martin, 2008) and (Kukich, 1992), de-
the problem (the latter in much more depth), but
neither consider the continuous process of user
input in the production and correction of errors.
Their spellchecking algorithms assume a com-
pleted document and a cessation of user input.
I propose a more intelligent text entry engine, in
which the errors that contribute to easily corrected
misspellings (by minimum edit distance methods,
for instance) will be incorporated into the model
of edit distances. For example, if a user mistypes
“aling,” the engine will quickly autocorrect it as
“align,” but if the user then mistypes “desing,”
the engine will remember the “ng” → “gn” edit
it performed before, and suggest “design” instead
of “dewing” (my naive iPad suggests “dewing”
even though I make that typo all the time). This
might be more like active learning, but as I under-
stand it, the difference between active learning and
semisupervised learning over a non-finite (contin-
uous?) data set is that active learning involves the
human user telling the machine what to learn, and
semisupervised learning entails the machine infer-
ring what to learn from the user’s normal behavior
(i.e. un-labeled data); my implementation would
be a combination of the two.
2 Colloquial/formal classification
This classifier will train on very large corpora that
are labeled at the global level; I will consider the
LDC’s switchboard corpus or the Simple English
language of Wikipedia as “colloquial,” and New
York Times or Wall Street Journal articles as “for-
mal.” These will be very noisy data sets, but I be-
lieve some traits of formality or colloquialism can
be teased from the documents. Given those two
poles of corpora, I will calculate the distance be-
tween my unlabeled, to-be-classified documents,
and documents in those corpora. (Do and Ng,
2006) present an improved algorithm for the “mul-
ticlass text classification task” that seems to be
suited to similar classification between such cor-
pora.1

References
Chuong Do and Andrew Ng. 2006. Transfer learning
for text classification. In Y. Weiss, B. Schölkopf,
and J. Platt, editors, Advances in Neural Information
Processing Systems 18, pages 299–306. MIT Press,
Cambridge, MA.
Daniel Jurafsky and James H. Martin, 2008. Speech
and Language Processing (2nd Edition), pages 72–
79. Prentice Hall, 2 edition.
Karen Kukich. 1992. Techniques for automatically
correcting words in text. ACM Comput. Surv.,
24(4):377–439.

1
The results section is somewhat opaque to me, but over-
all it seems like a reasonable place to start; it was not so easy
finding relevant articles for this idea.