QoE Fairness for Adaptive Video Streaming using Deep Reinforcement Learning

November 15, 2021 – ,

Topic Description

What is QoE fairness and how can it be achieved across multiple users and content types?

Quality of Experience (QoE) measures the perceived quality at the user end. It can be estimated with user studies and is often modelled in terms of Quality of Service (QoS) metrics. For example, the QoE of a video stream might drop if the bandwidth (QoS) is too low to handle the current resolution. Fairness can be defined in many different ways. It is often expressed in terms of guarantees, equality and equity.

Imagine multiple flows (e.g. video streams) that are forwarded through a network. The network has limited resources (e.g. available bandwidth, processing capabilities) that have to be distributed among the flows. The idea of guarantees is to ensure that each flow gets at least some proportion of the resources in order to provide at least a minimal level of functionality (user experience). Equality means that each flow gets the same share of resources, irrespective of the needs of the content type that is being transmitted (see TCP fairness [6]). With equity, the needs of the individual flows are taken into consideration when distributing the resources [4].

With this thesis, we want to study how to model and achieve fairness with different types of flows in the context of networking. It is also possible to consider flow transformations and different user requirements (e.g. different preferred video resolutions) instead. Each flow type should have its own stochastic utility function depending on the allocated resources. The aim is to achieve equal utility for all flows by learning the utility functions and then distributing the resources accordingly.

In this bachelor's or master's thesis, your tasks would be the following

  • Analyze related literature about fairness (in the context of network flows and QoE) [2]
  • Create a formal model of the problem, e.g. as a variant of the stochastic multi-armed bandit problem [0, 1, 3] with multi-dimensional continuous controls for the resource allocation (can also be discrete)
  • Select an appropriate approach to solve the problem (e.g. Reinforcement Learning [5])
  • Implement a prototype

    • bachelor's thesis: proof of concept
    • master's thesis: extended proof of concept with realistic scenario

  • Evaluation of the approach, e.g. in comparison with equality motivated by TCP fairness

 

Prerequisites

There are no hard requirements for this topic. However, we recommend

  • Background in (statistical) Machine Learning and Reinforcement Learning
  • Good programming skills in Python

 

Contact

If you are interested, please contact Jannis Weil (jannis.weil(at)kom.tu-darmstadt.de). Your email should include the following information:

  • Your course of studies
  • What interests you about this topic
  • Your prior experience in related topics, e.g. lectures

 

References

[0] Slivkins, Aleksandrs. "Introduction to multi-armed bandits." arXiv preprint arXiv:1904.07272 (2019)
[1] P. Auer, N. Cesa-Bianchi, and P. Fischer. "Finite-time analysis of the multiarmed bandit problem." Machine learning 47.2 (2002).
[2] T. Hoßfeld, L. Skorin-Kapov, P. E. Heegaard, and M. Varela, “Definition of QoE Fairness in Shared Systems,” IEEE Communications Letters, vol. 21, no. 1, Art. no. 1, 2017.
[3] V. Patil, G. Ghalme, V. Nair, and Y. Narahari, “Achieving Fairness in the Stochastic Multi-Armed Bandit Problem,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, Art. no. 04, 2020.
[4] M. Seufert, N. Wehner, and P. Casas, “A Fair Share for All: TCP-Inspired Adaptation Logic for QoE Fairness Among Heterogeneous HTTP Adaptive Video Streaming Clients,” IEEE Transactions on Network and Service Management, vol. 16, no. 2, Art. no. 2, 2019.
[5] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press, 2018. Available: incompleteideas.net/book/the-book.html.
[6] H. A. Wang and M. Schwartz, “Achieving Bounded Fairness for Multicast and TCP Traffic in the Internet,” SIGCOMM Comput. Commun. Rev., vol. 28, no. 4, 1998.

Supervisors

Jannis Weil (KOM) and Anam Tahir (SOS)

Keywords:

Research Area(s):

Tutor: Weil,

Student: Julian Barthel

Completed Theses