Using Machine Learning for Big Data Systems

September 04, 2017 – /,


Future Internet will be connecting billions of heterogeneous devices producing terabytes of data per second. An example of such an emerging class of applications that connects massive amount of devices are Internet of Things (IoT) applications. To comprehend this large amount of data traffic with real-time guarantees there is a need of a strong data processing paradigm. Complex Event Processing (CEP) is an emerging paradigm that provides this by obtaining higher-level information from low-level data through operator graphs, which jointly represent the interest of the user. This is accomplished mainly by placement and distribution of CEP operators inside the network. But the placement of the operators readily becomes inefficient because of the frequent changes in the environmental context, e.g., uniform or bursty traffic. This also impacts desired quality levels required by the application. CEP reacts to this by reconfiguration of operators and its mechanisms in the network. But to achieve the desired level of quality, proactive reconfigurations must be in place. In this thesis, we explore methods that can provide this proactivity in CEP mechanisms.  

If this topic* interests you, please get in touch with anyone of us. 

Main tasks:

§  Literature research on methods for predicting environmental conditions (context) particularly using machine learning methods in Hidden Markov Model (HMM).

§  Mathematical formulation of the CEP context prediction as a state prediction problem in HMM.

§  Design of the HMM prediction module in an adaptive CEP system for proactive transitions of CEP mechanisms.

§  Implementation and evaluation of the aforementioned design w.r.t. the fulfilment of quality levels specified by application end user.

[1] Ghahramani, Z. (2001). An Introduction to Hidden Markov Models and Bayesian Networks. World Scientific Publishing Co., Inc., 1, 9–42.

[2] Weisenburger, P., Luthra, M., Koldehofe, B., & Salvaneschi, G. Quality-Aware Runtime Adaptation in Complex Event Processing. SEAMS ‘17

Required profile:

§  Interest in communication networks and probability theory.

§  Good programming skills in Scala (or Java) and MATLAB.

download corresponding tendering

Keywords: Complex Event Processing, Machine Learning, Proactive Transitions

Research Area(s): Self-organizing Systems & Overlay Communications

Tutor: Manisha Luthra, Sounak Kar

Open Theses