HANNA: A Tool for Hardware Prototyping and

Benchmarking of ANNs
J avier Garrigs, J os J . Martnez, J avier Toledo and J os M. Ferrndez
Dpto. Electrnica, Tecnologa de Computadoras y Proyectos - Universidad Politcnica de Cartagena
C/. Doctor Fleming s/n, 30202 Cartagena - Spain
This research is being funded by Ministerio de Ciencia y Tecnologa TIC 2003-09557-C02-02.
1. Introduction
For some applications, designers must implement an ANN model over different
platforms to meet performance, cost or power constrains, a process still more
painful when several hardware implementations have to be evaluated. Continuous
advances in VLSI technologies, computer architecture and software development
make it difficult to find the adequate implementation platform. HANNA (Hardware
ANN Architect), is a tool designed to automate the generation of hardware
prototypes of MLP-like neural networks over FPGA devices. Coupled with
traditional Matlab/Simulink environments the model can be synthesized,
downloaded to the FPGA and co-simulated with the software version to trade off
area, speed and precision requirements.
IWINAC2007
2nd. INTERNATIONAL WORK-CONFERENCE on the INTERPLAY between NATURAL
and ARTIFICIAL COMPUTATION
2. Software/Hardware prototyping
System Generator
TM
for DSP (Xilinx Inc.), a plug-in to the Simulinkenvironment
(The Mathworks Inc.) designed to develop high-performance DSP systems for
FPGAs. Advantages are:
3. Proposed ANN Hardware Architecture
9
8
Out8
7
Out7
6
Out6
5
Out5
4
Out4
3
Out3
2
Out2
1
Out1
xlslice [a:b]
SliceOEN5
xlslice [a:b]
SliceOEN4
xlslice [a:b]
SliceOEN3
xlslice [a:b]
SliceOEN2
xlslice [a:b]
SliceCNT5
xlslice [a:b]
SliceCNT4
xlslice [a:b]
SliceCNT3
xlslice [a:b]
SliceCNT2
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
R t
Out1
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL5N8
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL5N7
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL5N6
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL5N5
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL5N4
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL5N3
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL5N2
In2
I 3
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL5N1
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
I 17
Out1
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
In17
In18
In19
In20
In21
Reset
cnt
oen
Out1
NeuronL4N8
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
In17
In18
In19
In20
In21
Reset
cnt
oen
Out1
NeuronL4N7
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
In17
In18
In19
In20
In21
Reset
cnt
oen
Out1
NeuronL4N6
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
In17
In18
In19
In20
In21
Reset
cnt
oen
Out1
NeuronL4N5
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
In17
In18
In19
In20
In21
Reset
cnt
oen
Out1
NeuronL4N4
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
In17
In18
In19
In20
In21
Reset
cnt
oen
Out1
NeuronL4N3
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
In17
In18
In19
In20
In21
Reset
cnt
oen
Out1
NeuronL4N2
In2
I 3
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
In17
In18
In19
In20
In21
Reset
cnt
oen
Out1
NeuronL4N1
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
Reset
cnt
oen
Out1
NeuronL3N9
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
Reset
cnt
oen
Out1
NeuronL3N8
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
Reset
cnt
oen
Out1
NeuronL3N7
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
Reset
cnt
oen
Out1
NeuronL3N6
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
Reset
cnt
oen
Out1
NeuronL3N5
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
Reset
cnt
oen
Out1
NeuronL3N4
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
Reset
cnt
oen
Out1
NeuronL3N3
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
Reset
cnt
oen
Out1
NeuronL3N2
In2
I 3
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
Reset
cnt
oen
Out1
NeuronL3N1
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
R t
Out1
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL2N8
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL2N7
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL2N6
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL2N5
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL2N4
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL2N3
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL2N2
In2
I 3
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
In16
Reset
cnt
oen
Out1
NeuronL2N1
xlhdlcosim
ModelSim
ModelSim
xlblackbox2
Control
9
In8
8
In7
7
In6
6
In5
5
In4
4
In3
3
In2
2
In1
1
Reset
1
Out1
xlcast force
Reinterpret2
xlcast force
Reinterpret1
xlcast force
Reinterpret Bias
xlblackbox2 vout
dinput
weight
cnt
oen
reset
Neuron
0
0
Constant2
0
Constant1
hi
xlconcat
hi
lo
cat
ConcatW2
xlconcat
hi
lo
cat
ConcatW1
hi
xlconcat
hi
lo
cat
ConcatIn8
xlconcat
hi
lo
cat
ConcatIn7
xlconcat
hi
lo
cat
ConcatIn6
xlconcat
hi
lo
cat
ConcatIn5
xlconcat
hi
lo
cat
ConcatIn4
xlconcat
hi
lo
cat
ConcatIn3
xlconcat
hi
lo
cat
ConcatIn2
xlconcat
hi
lo
cat
ConcatIn1
127
Bias
7
In8
6
In7
5
In6
4
In5
3
In4
2
In3
1
In2
Reset
In1
In2
In3
In4
In5
In6
In7
In8
In9
In10
In11
In12
In13
In14
In15
Out1
Out2
Out3
Out4
Out5
Out6
Out7
Out8
Out9
Out10
Neural Network
System
Generator
Hardware-ANN implementation of the XOR function
and comparation with the software ANN implementation
an the logical function (1)
(2)
(3)
Y
X
.75
W8
.75
W7
-1
W6
.75
W5
.75
W4
-.5
W3
-.75
W2
-.75
W1
1
W0
Scope
XOR
Logical
Operator
Reset
In1
In2
Out1
Hardware
Neural Network
fpt dbl
GatewayOut
dbl fpt
GatewayIn2
d
b
l
fp
t
GatewayIn1
d
b
l
fp
t
GatewayIn
127
Gain1
127
Gain
0
Constant3
1
Constant2
1
Constant1
1
Constant
bias
W0
X
W1
Y
W2
Z
ANN_OR
bias
W0
X
W1
Y
W2
Z
ANN_NAND
bias
W0
X
W1
Y
W2
Z
ANN_AND
System
Generator
6. Conclusions
HANNA is a tool for hardware prototyping of MLP-like ANNs over programmable
devices. The benefits offered by system level design tools, like the suite
Matlab/Simulink/SystemGenerator that allows targeting multiple implementation
platforms, can be further extended with tools like HANNA in case of FPGA
prototyping, allowing a high level (neuron level) description of the ANN, instead of
the standard lower-level RTL description of the network. The tool and our design
methodology have been validated through two examples, resulting in hardware
design times reduced to minutes and acceptable performance over current FPGA
devices.
Future work involves increasing the capabilities of HANNA by including learning
mechanisms, a microprocessor-based network controller for sophisticated
execution schemes, and taking advantage of FPGAs dynamic reconfiguration
features for on-line network evolution. HANNA is available upon request for non-
profit research institutions and organizations.
5. Results for XOR and speaker recognition applications
1
.
3. HANNA capabilities
HANNA is a set of Matlab scripts,
primitives templates and a GUI.
Starting from users specifications
and proper templates, VHDL for the
net, control and neuron blocks is
generated. After network generation,
a Simulink model file is created which
contains the network architecture,
and a second file with same name
and sufix _vars is generated to store
workspace and general configuration
parameters. The model generated
includes a synthesis block configured
for the custom tools and hardware
available for circuit implementation,
shown in Figure 3.(a).
MUX
WEIGHT
ROM
SYNCHR. &
CONTROL

+
*
DATA
INPUTS
DATA
INDEX
NEURON
OUTPUT
Fig.3. (a) Standard neuron transfer functionfor MLPs, (b) Neuronarchitecture, definedas highlyparametrizable VHDL.
Fig.4. Net Architecture. (a) Net block (bottom) and vhdl synthesis block (top), (b) neuron layers
(bottom) and control logic (top), (c) input casting (left) and neuron block (right).
(a)
(b)
(c)
Fig.1. Net Architecture (right.
high-level schematic flow,
powerful Matlab visualization tools
for test-benches,
successive refined software
versions developed with reduced
additional effort, each one
introducing more hardware friendly
characteristics, to observe the
effect of latency, quantization, etc.
hardware in the loop capability.
Hardware version of the ANN can
be developed and validated against
the previous versions.
As an example, Figure 1
demonstrate implementation and
co-simulation of three different
versions of the XOR function: using
the standard logical operator
(green), a software full-precision
ANN (blue), and its hardware
quantized- version (orange).
(a)
(b)
However, using System Generator blockset, the hardware model has to be defined
at the RTL, which requires certain knowledge of digital and logic design and the
effort of generating a detailed description of the ANN algorithm. HANNA is
intended to automatically generate a synthesizable System Generator description
starting from just a high level definition of the network. It takes then minutes to
have the hardware version downloaded and working on a FPGA prototyping
board. Moreover, software and hardware outputs can be simultaneously verified
with the same, unique, test bench.
Fig.2. HANNA GUI showing some network parameters.
1
Individually differentiating every member in a group of 11 spanishspeakers.
Results showed 500x/800x performance boost over Matlab (P4 3.2GHz, 2GB RAM).