Research Topic and Initial Reading List

Roberto Camacho B.
September 4, 2015

Main Area of Research Interest

My main area of research interest is computation offloading. Computation
offloading is seen as a possible solution to the inherent limitations of mobile
computing, such as limited processing and battery power. The motivation
is that future computationally-intensive applications such as augmented reality, or real-time image and speech processing, will require more computational power than the available on a mobile device, or a very high-energy
consumption.
The proposed solution is to offload these applications to surrogate devices, which should have better computing resources and preferably unlimited power available. The limitations of this approach are clear: the input
and output data must be sent through a network connection between the
local device and the surrogate. If the network conditions are not ideal, or if
there is a lot of data to be transferred, the application should probably not
be offloaded.
Our current focus is on the offloading decision. We need to decide
whether offloading would be beneficial, and in case several surrogates are
available, which one to offload to. To take this decision, several approaches
have been studied, but most of them are based on completion time estimation. There are several different techniques for completion time estimation,
including parametric and non-parametric statistical analyses, code profiling,
benchmarking and other approaches that combine these methods.
In the context of the offloading decision, an estimation method should be
accurate, but should also have a very fast response time. Therefore, a balance between speed and accuracy must be met to reduce the overhead of the
prediction while selecting the device that will provide the best performance.
My research, and possible dissertation topic, is oriented into finding if
there is a correlation between the type of an application (with regards to
the computation and memory access patterns) and the best estimation
method. This will be accomplished by using machine learning classification
methods to cluster the applications per type, and then analyze the performance of different prediction methods (including simple heuristics and
machine learning algorithms) to select the best for each cluster. I expect to
significantly reduce the required time to get an estimation so that this can
be actually implemented in a scheduler.
My previous experience includes my Masters thesis, were I used different
estimation techniques to obtain the best configuration for several applications executed on the Intel Xeon Phi, a manycore device which has many
different variables that can be set at runtime and which have an effect on
1

performance, which is a similar problem but from a different perspective.

Initial Reading List

[1] proposes a method to estimates the resources required by a job in
a grid computing environment, based on historical information. This
method is implemented as part of a grid scheduler. This publication
was found on the Caltech CODA repository.
[2] shows how to take into account virtualization for performance estimation. This is significantly important for my research since the focus
will probably be in virtual environments such as Android. This paper
was published in the Proceedings of IEEE Computer Societys Annual
International Symposium on Modeling, Analysis, and Simulation of
Computer and Telecommunications Systems.
[3] proposes an online method for execution time prediction of applications, for which execution progress can be collected at run-time. The
collected data is extrapolated using predefined evolution models. This
article was published in The Journal of Supercomputing.
[4] presents an approach for prediction of MPI-based parallel applications using static code analysis and trace-based simulation. This
paper appears on The Journal of Supercomputing.
[5] introduces a statistical method to predict CPU time, I/O and memory usage for an application. The first step in this case involves statistical clustering to identify high-density regions of process resource
usage. Based on these regions a state-transition model is created to
accurately predict the resource usage. This approach is very interesting for my research because it also involves statistical clustering. This
paper was published in IEEE Transactions on Software Engineering.
[6] presents a framework that uses Kernel Canonical Correlation Analysis (KCCA) to predict the execution time of MapReduce jobs, which
is not exactly the same problem, but the estimation method seems
to apply to more general problems. Published in Data Engineering
Workshops (ICDEW), 2010 IEEE 26th International Conference
[7] is a project report that focuses on prediction of execution time
for parallel jobs using historical information. The historical records

are clustered into classes based on the execution time, and the average euclidean distance is used to predict the execution time. Various
machine learning algorithms were also implemented to compare the
accuracy of the prediction.
[8] is important to my research because they developed a system that
automatically extracts a large number of features from program execution using Sparse Polynomial regression, which I will might try to use
to perform feature selection. This paper was published in Advances in
Neural Information Processing Systems 23.
[9] presents methods based in time delay neural network and regression
for prediction of workload in Grid and Cloud platforms. This paper
was published in the Proceedings of the International Conference on
Computer and Information Technology.
[10] presents a method which combines analytic benchmarking and
statistical analysis techniques. The benchmarking is similar to the
training phase of a machine learning algorithm. This paper was published in the IEEE Heterogeneous Computing Workshop.
[11] shows a model created using the least-squares method with the
results obtained from the measurement of various configurations. This
article was published in the Journal of Parallel Computing.
[12] presents a framework called Mantis which is used to predict the
resource consumption of Android applications. This framework uses
machine learning and program analysis techniques to obtain accurate
predictions. Several works on computational offloading have used this
framework to obtain a prediction. This paper was published on the
IEEE Transactions on Mobile Computing.
[13] proposes a method to predict execution time for Grid applications
using machine learning models based on multilayer perceptron. The
authors also use a principal feature selection algorithm to select the
most important runtime features for prediction. This paper appears
in the Proceedings of the 8th Workshop on Workflows in Support of
Large-Scale Science.
[14] uses machine learning algorithms to predict resource usage, with
a tree-based approach named PQR2. It also provides several good
references. This article was published in the IEEE/ACM International
Conference on Cluster, Cloud and Grid Computing.
3

 [15] is from the same author of [14], however this is a dissertation and
contains a longer explanation of their method. Many more papers are
referenced. The only negative of these two publications is that they
are both from 2010, so it is important to look in the Cited by
[16] presents a method to estimate execution time in parallel devices.
This method is based in code profiling, to estimate the percentage
of parallel regions on a program, and Amdahls Law, to predict the
performance of this application for a different machine with a different
number of cores. This paper was presented in the Proceedings of the
ACM/SPEC international conference on performance engineering.
[17] introduces a method to predict execution time of applications
using an historical records and an instance-based learning technique,
which is one of the techniques I want to implement in my research
since it is benefited from clustering. This article was published by
NASA.
[18] presents a mathematical approach to predict the expectation of the
maximum execution time of all processors on a parallel system, since
this will be the total execution time for a parallel application. This
paper is presented in IEEE Transactions on Parallel and Distributed
Systems.
[19] proposes an approach based on augmented code profiling and
benchmark analysis used to generate Representative Set of Templates (RST) which represent the execution behavior of a task with
different inputs. This paper was presented in the International Conference on Parallel Processing.
[20] uses parametric code profiling and parametric benchmarking techniques. It also incorporates the concept of an architecture independent computation model to estimate the execution times. This is a
very different approach but reading should be useful for the Previous
Work section. This paper was published in the IEEE Heterogeneous
Computing Workshop.

Bibliography
[1] Ali Arshad, Ashiq Anjum, Julian Bunn, Richard Cavanaugh, Frank Van
Lingen, Muhammad Atif Mehmood, Harvey Newman, Conrad Steenberg, and Ian Willers. Predicting the Resource Requirements of a Job
Submission. Computing in High Energy Physics, pages 750753, 2004.
[2] Fabian Brosig, Fabian Gorsler, Nikolaus Huber, and Samuel Kounev.
Evaluating approaches for performance prediction in virtualized environments. Proceedings - IEEE Computer Societys Annual International Symposium on Modeling, Analysis, and Simulation of Computer
and Telecommunications Systems, MASCOTS, pages 404408, 2013.
[3] Maria Chtepen, Filip HA Claeys, Bart Dhoedt, Filip De Turck, Jan
Fostier, Piet Demeester, and Peter A Vanrolleghem. Online execution
time prediction for computationally intensive applications with periodic
progress updates. The Journal of Supercomputing, 62(2):768786, 2012.
[4] Bogdan Florin Cornea and Julien Bourgeois. A framework for efficient
performance prediction of distributed applications in heterogeneous systems. The Journal of Supercomputing, 62:16091634, 2012.
[5] Murthy V Devarakonda and Ravishankar K Iyer. Predictability of process resource usage: A measurement-based study on unix. Software
Engineering, IEEE Transactions on, 15(12):15791586, 1989.
[6] Archana Ganapathi, Yanpei Chen, Armando Fox, Randy Katz, and
David Patterson. Statistics-driven workload modeling for the cloud.
pages 8792, 2010.
[7] Gowthami Manogna Gottipati. Predicting Runtimes for Production
Parallel Jobs. Master, Indian Institute of Science, 2012.

[8] Ling Huang, Jinzhu Jia, Bin Yu, Byung-Gon Chun, Petros Maniatis,
and Mayur Naik. Predicting execution time of computer programs using
sparse polynomial regression. pages 883891, 2010.
[9] Md Toukir Imam, Sheikh Faisal Miskhat, Rashedur M. Rahman, and
M. Ashraful Amin. Neural network and regression based processor
load prediction for efficient scaling of grid and cloud resources. 14th
International Conference on Computer and Information Technology,
ICCIT 2011, (Iccit):333338, 2011.
[10] Michael A Iverson, Fusun Ozguner, and Lee C Potter. Statistical prediction of task execution times through analytic benchmarking for scheduling in a heterogeneous environment. pages 99111, 1999.
[11] Yoshinori Kishimoto and Shuichi Ichikawa. Optimizing the configuration of a heterogeneous cluster with multiprocessing and execution-time
estimation. Parallel Computing, 31(7):691710, 2005.
[12] Yongin Kwon, Sangmin Lee, Hayoon Yi, Donghyun Kwon, Seungjun
Yang, Byung-gon Chun, Ling Huang, Petros Maniatis, Mayur Naik,
and Yunheung Paek. Mantis: Efficient Predictions of Execution Time,
Energy Usage, Memory Usage and Network Usage on Smart Mobile
Devices. IEEE Transactions on Mobile Computing, 6(1):11, 2014.
[13] Muhammad Junaid Malik, Thomas Fahringer, and Radu Prodan. Execution time prediction for grid infrastructures based on runtime provenance data. In Proceedings of the 8th Workshop on Workflows in Support of Large-Scale Science - WORKS 13, pages 4857, 2013.
[14] Andrea Matsunaga and Jose AB Fortes. On the use of machine learning
to predict the time and resources consumed by applications. pages 495
504, 2010.
[15] Andrea Matsunaga. Automatic enablement, coordination and resource
usage prediction of unmodified applications on clouds. PhD thesis, University of Florida, 2010.
[16] Achille Peternier, Walter Binder, Akira Yokokawa, and Lydia Chen.
Parallelism profiling and wall-time prediction for multi-threaded applications. pages 211216, 2013.
[17] Warren Smith and Parkson Wong. Resource selection using execution
and queue wait time predictions. NASA Ames Research Center TR
NAS-02-003, 2002.
6

[18] Junqing Sun and Gregory D Peterson. An effective execution time

approximation method for parallel computing. Parallel and Distributed
Systems, IEEE Transactions on, 23(11):20242032, 2012.
[19] Jaehyung Yang, Ishfaq Ahmad, and Arif Ghafoor. Estimation of execution times on heterogeneous supercomputer architectures. 1:219226,
1993.
[20] Jaehyung Yang, A Khokhar, S Sheikh, and A Ghafoor. Estimating
execution time for parallel tasks in heterogeneous processing (hp) environment. pages 2328, 1994.