Pratyush Agnihotri

Motivation:

Internet of Things describes a new world of heterogeneous objects such as sensors, smartphones, actuators etc, which intelligently interacts and communicate with each other and offer a better quality of life. With the increase of these objects, the data volume is growing as well. Processing this volume of data can be helpful to various application domains to reveal complex patterns, gain fast insights or react to observed situations, e.g., fraud detection, smart factories, telecommunication, etc. However, it becomes extremely challenging to continuously process this high volume of data in a timely manner and meet these applications' demands. One of the possible ways can be to use heterogeneous resources' capability such as GPUs, FPGAs, etc. to accelerate the big data analytics instead of relying on homogeneous resources, e.g., CPU.

Task:

• Analyze existing related literature.
• Formalizing the problem and developing the solution and
• Setting up development environment and implementing the proof of concept or prototype.
• Evaluation of the proposed solution in comparison to the existing approaches. .

Prerequisites:

There are no hard guidelines for this topic. However, it is preferable if you have:

• Good programming skills in Java and/or Scala.
• Understanding of the concept of .
• communication networks and .
• big data processing engines, e.g., Apache Flink, Kafka.

Contact:

I am looking for motivated students interested in working on cutting-edge technologies and IoT applications area. If you are interested in writing a Bachelor's or Master's Thesis or gaining experience in MAKI project as HiWi, please feel free to contact me. You can email me the following information:

• Your CV or small text about your courses and prior experiences in this area.
• The short note on your motivation about this topic and,
• Transcript or TUCaN grade list.

Motivation

 In Distributed Stream Processing Systems (DSPS), queries are usually long-running. They typically deal with a very high workload of million or even billion of events per second. Under such scenarios, parallelism plays an important role in providing scalability for DSPS. One of the core
decisions here is to determine the right parallelization degree to process a high load of events and queries while ensuring high throughput and low latency requirements. In this context, parallelization can be controlled by defining the degree of parallelism for each operator in the operator graph.
The main questions we want to investigate here are - 

•  What is the right parallelism degree for query processing? (what is the performance in terms of end-to-end latency for parallel stream processing? At what extend the model can be used - seen and unseen behaviour? What are the limitations of models? At which can be extended?)
• How to parallelize the processing of DSPs operators? 

 Prerequisites

 There are no hard guidelines for this topic. However, it is preferable if you have:

• Good programming skills in Java and Python
• Good understanding of the concepts of machine learning
• Understanding of the concept of communication networks, big data processing engines, e.g., Apache Flink, Kafka. 

 Reference Literature

• Three steps is all you need: fast, accurate, automatic scaling decisions for distributed streaming dataflows.
• A Comprehensive Survey on Parallelization and Elasticity in Stream Processing
• Resource Management and Scheduling in Distributed Stream Processing Systems: A Taxonomy, Review, and Future Directions
• One Model to Rule them All: Towards Zero-Shot Learning for Databases Benjamin
• Apache Flink documentation: nightlies.apache.org/flink/flink-docs-master/docs/learn-flink/overview/ [https://nightlies.apache.org/flink/flink-docsmaster/docs/learn-flink/overview/] 

Contact:

I am looking for motivated students interested in working on cutting-edge technologies and IoT applications area. If you are interested in writing a Bachelor's or Master's Thesis or gaining experience in MAKI project as HiWi, please feel free to contact me. You can email me the following information:

• Your CV or small text about your courses and prior experiences in this area.
• The short note on your motivation about this topic and,
• Transcript or TUCaN grade list.

Distributed Stream Processing Systems (DSPS) allow real-time processing of data to fetch meaningful information. Big giants like Twitter, Facebook, and Alibaba rely on DSP for real-time data analytics. For example, click analytics, credit card fraud detection, etc. In DSPS, queries are usually long-running. They typically deal with a very high workload of millions or even billions of events per second. Under such scenarios, parallelism plays an important role in providing scalability for DSPS where either the input stream is partitioned into multiple processes or sub-streams or duplicated to be processed by operators and their instances. However, existing DSPS lacks the benchmarking of the parallel processing capability of operators under various workloads, event rates, and different queries.

Research Goals:

• Literature study of benchmarking parallel stream processing.
• Implementing Window and Pane-based parallelization strategy.
• Evaluating the performance for various benchmarks
• Displaying the performance evaluation result on the front end.
• Possible DSPS to be used: Apache Flink