A Collection of Website Benchmarks Labelled For (T) Emplate Detection and Content Extraction

A Collection of Website Benchmarks Labelled for
Template Detection and Content Extraction

Julian Alarte, David Insa, Josep Silva, Salvador Tamarit
MiST Research Group, Universitat Polit`
ecnica de Val`
encia
and
Babel Research Group, Universidad Polit
ecnica de Madrid
XV Jornadas Sobre Programaci

on y Lenguajes (PROLE15)
September 15th, 2015
Context and Motivation

Template Detection
Identifies the template of a webpage.
Essential for indexing tasks:
I
I
Templates represent between 40% and 50% of data on the Web

Usually contain irrelevant information (e.g. advertisements, menus and
banners)
Avoids waste of resources (storage space, bandwidth, etc.)

Important tool for website developers and analyzers.
Alarte, Insa, Silva, Tamarit (UPV & UPM)
TECO Benchmark Suite
(PROLE15)
1 / 12

Template Detection
Identifies the template of a webpage.
Essential for indexing tasks:
I
I
Templates represent between 40% and 50% of data on the Web

Usually contain irrelevant information (e.g. advertisements, menus and
banners)
Avoids waste of resources (storage space, bandwidth, etc.)

Important tool for website developers and analyzers.
Content Extraction
Identifies the main content of the webpage.
Essential for many information retrieval and processing tasks.
(PROLE15)
1 / 12

Template Detection & Content Extraction
Two of the main areas of information retrieval applied to the Web.
Complementary: main content is not part of the template.
(PROLE15)
2 / 12
Benchmark Suite for Template Detection

and Content Extraction
Testing, Comparing and Tuning

Collections of heterogeneous benchmarks: ensures generality of the
techniques
Gold standard: ensures the same evaluation criteria.
Using a benchmark suite

Training phase: to optimize the techniques by adjusting parameters
Evaluation phase: to measure the performance with objective criteria.
They need disjoint sets of webpages.
(PROLE15)
3 / 12

Previous Situation
Lack of a public and neutral benchmark suite
Evaluations:
I
I
I
with dierent benchmarks

with dierent kinds of templates
using dierent criteria
Results hardly comparable with other techniques.
(PROLE15)
4 / 12

Our Experience
Initial intention: use a public benchmark suite, CleanEval
I
Widely used in the literature
(PROLE15)
5 / 12

Our Experience
I
I

Not prepared for template detection
(PROLE15)
5 / 12

Our Experience
I
I

Second option: Contacted the authors of other techniques
(PROLE15)
5 / 12

Our Experience
I
I


I
We could not use their benchmarks due to:

F
F
F
privacy
copyright
unavailability
(PROLE15)
5 / 12

Our Experience
I
I


I

F
F
F
privacy
copyright
unavailability
Final choice: Build our own free and publicly available benchmark suite.
(PROLE15)
5 / 12
The TECO Benchmark Suite

Our approach
TECO (TEmplate detection and COntent extraction benchmarks suite)
Consists in:
I
I
I
Benchmark suite
Gold Standard
Scripts to automatize the benchmarking process
Scope:
I
I
Template detection
Content extraction
Goal:
I
I
I
Test
Compare
Tune
Uses:
I
I
Training
Evaluation
(PROLE15)
6 / 12

Features
Result of a research project:
I
I
A new technique for content extraction

Later adapted for template detection.
40 real heterogeneous websites downloaded from Internet.

Open, extensible, publicly available and free.
Webpages in dierent languages: to test language-independent features.
Downloading of the webpages:
I
I
All needed elements for correct visualization: HTML, images, scripts, CSS...
SiteSucker (OS X) and wget (Linux).
Each benchmark is composed of:

1
2
Key page. Target webpage.

All those webpages (from the same website) that are linked by the key page as
well as the webpages linked by them.
Gold standard (for each key page) using labels:

I
HTML classes notTemplate and mainContent.
(PROLE15)
7 / 12
Producing the Gold Standard
Four dierent engineers

Independently:
I
I
Manually explored the key page and the webpages accessible from it
Choose what part of the webpage is the template and what part is the main
content.
Together:
I
Same actions sharing their individual opinions.
(PROLE15)
8 / 12
Benchmark Classification
Classification 1: All benchmarks have been classified into five groups:
Companies / Shops,
Forums / Social,
Personal websites / Blogs,
Media / Communication,
Institutions / Associations.
www.bbc.co.uk/news/index.html (Media / Communication)
Classification 2: All benchmarks have been classified according to their size and
the proportion of their template / main content.
Id
24
Benchmark
www.bbc.co.uk/news/index.html
Nodes
2991
T. Nodes
364
M.C. Nodes
1360
Classification 3: The benchmarks were also classified according to the number of

webpages that implement the template.
Id
24
VL
5
TT
0
PT
5
DT
0
Notes (peculiarities)
Several templates (but very similar).
(PROLE15)
9 / 12
Downloading and using the suite
Download
Directory with 40 folders.
Scripts to automatize the
benchmarking process
Rules for using the suite

1
Publish the results so that they are

publicly available.
Provide enough information so that

anyone can easily duplicate the
experiments.
http://users.dsic.upv.es/~jsilva/retrieval/teco/
(PROLE15)
10 / 12
Rules for extending the suite
Websites included in TECO must be real and online websites not created by
the people who submit the benchmark.
All benchmarks must be localized, so all resources are accessible oine.
Each benchmark must be composed of a webpage and at least all webpages

accessible from it with two clicks.
All benchmarks must be manually reviewed by at least two people before

being submitted.
All benchmarks submitted must be signed.
Researchers must follow the labeling guidelines of TECO.
All benchmarks submitted should not have a direct relation with a particular
technique or tool.
(PROLE15)
11 / 12
Conclusions & Future Work



Our Experience
I
I


I

F
F
F
privacy
copyright
unavailability
Septembe 15th, 2015
(PROLE15)
2 / 12
Septembe 15th, 2015
(PROLE15)
5 / 12
Our approach
Consists in:
I
I
I
Download
Benchmark suite
Gold Standard

Scope:
I
I
Template detection
Content extraction
Goal:
I
I
I
Test
Compare
Tune
Uses:
I
I
Training
Evaluation

publicly available.

experiments.
Septembe 15th, 2015
(PROLE15)
6 / 12
Septembe 15th, 2015
(PROLE15)
10 / 12
(PROLE15)
12 / 12
Conclusions & Future Work



Our Experience
I
I


I

F
F
F
privacy
copyright
unavailability
Septembe 15th, 2015
(PROLE15)
2 / 12
Septembe 15th, 2015
(PROLE15)
5 / 12
Our approach
Consists in:
I
I
I
Download
Benchmark suite
Gold Standard

Scope:
I
I
Template detection
Content extraction
Goal:
I
I
I
Test
Compare
Tune
Uses:
I
I
Training
Evaluation

publicly available.

experiments.
Septembe 15th, 2015
(PROLE15)
6 / 12
Septembe 15th, 2015
(PROLE15)
10 / 12
Ongoing Extension (TECO 2.0)

Includes 90 benchmarks (50 more than TECO 1.0).
Contains explicit information about subtemplates.
(PROLE15)
12 / 12
A Collection of Website Benchmarks Labelled for

Template Detection and Content Extraction
Julian Alarte, David Insa, Josep Silva, Salvador Tamarit
MiST Research Group, Universitat Polit`
ecnica de Val`
encia
and
Babel Research Group, Universidad Polit
ecnica de Madrid
XV Jornadas Sobre Programaci

on y Lenguajes (PROLE15)

A Collection of Website Benchmarks Labelled For (T) Emplate Detection and Content Extraction

Cargado por

Información del documento

Título original

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

A Collection of Website Benchmarks Labelled For (T) Emplate Detection and Content Extraction

Cargado por

Copyright:

Formatos disponibles

A Collection of Website Benchmarks Labelled for

Template Detection and Content Extraction

XV Jornadas Sobre Programaci

Context and Motivation

Templates represent between 40% and 50% of data on the Web

Avoids waste of resources (storage space, bandwidth, etc.)

Alarte, Insa, Silva, Tamarit (UPV & UPM)

TECO Benchmark Suite

September 15th, 2015

Context and Motivation

Templates represent between 40% and 50% of data on the Web

Avoids waste of resources (storage space, bandwidth, etc.)

Alarte, Insa, Silva, Tamarit (UPV & UPM)

TECO Benchmark Suite

September 15th, 2015

Context and Motivation

Alarte, Insa, Silva, Tamarit (UPV & UPM)

TECO Benchmark Suite

September 15th, 2015

Benchmark Suite for Template Detection

Testing, Comparing and Tuning

Using a benchmark suite

Alarte, Insa, Silva, Tamarit (UPV & UPM)

TECO Benchmark Suite

September 15th, 2015

Benchmark Suite for Template Detection

with dierent benchmarks

Results hardly comparable with other techniques.

Alarte, Insa, Silva, Tamarit (UPV & UPM)

TECO Benchmark Suite

September 15th, 2015

Benchmark Suite for Template Detection

Widely used in the literature

Alarte, Insa, Silva, Tamarit (UPV & UPM)

TECO Benchmark Suite

September 15th, 2015

Benchmark Suite for Template Detection

Widely used in the literature

Alarte, Insa, Silva, Tamarit (UPV & UPM)

TECO Benchmark Suite

September 15th, 2015

Benchmark Suite for Template Detection

Widely used in the literature

Second option: Contacted the authors of other techniques

Alarte, Insa, Silva, Tamarit (UPV & UPM)

TECO Benchmark Suite

September 15th, 2015

Benchmark Suite for Template Detection

Widely used in the literature

Second option: Contacted the authors of other techniques

We could not use their benchmarks due to:

Alarte, Insa, Silva, Tamarit (UPV & UPM)

TECO Benchmark Suite

September 15th, 2015

Benchmark Suite for Template Detection

Widely used in the literature

Second option: Contacted the authors of other techniques

We could not use their benchmarks due to:

Alarte, Insa, Silva, Tamarit (UPV & UPM)

TECO Benchmark Suite

September 15th, 2015

The TECO Benchmark Suite

Alarte, Insa, Silva, Tamarit (UPV & UPM)