A Low-Latency Edge-Cloud Serverless Computing Framework with a Multi-Armed Bandit Scheduler

Recently, serverless computing, particularly the Function-as-a-Service (FaaS) programming model, has become an important emerging technology for developers and cloud providers. It relieves developers from the burden of explicitly managing the computing resources and provides more accurate billing of the exact service and execution time. However, as a side effect of the service-side resource management, the system often inactivates a set of execution dockers, introducing a significant cold start time for the following invocation, resulting in unpredictable latency. Existing solutions mainly rely on improving an end-point management policy or scheduling into other same-tier endpoints and, more recently, considering a promising but simplified edge-cloud tier with available management information. The latter can mitigate latency by relying on offloading to a resource-rich cloud. In this paper, we consider extending the two-tier edge-cloud approach to not rely on any management information from the service side or account for function-dependent communication latency but to rely on a scheduler based on the multi-armed bandit (MAB) upper confidence bound (UCB) algorithm that dynamically learns from the prevailing real-time conditions to choose the best cloud platform to execute functions with minimal latency. A test bed was implemented, comprising an OpenWhisk system deployed on a local Kubernetes cluster (kind) and two commercial FaaS systems: Amazon Web Services (AWS) Lambda and Google Cloud Functions (GCF). The scheduler was tested in real-time using the serverless benchmark suite (SeBS). Our results show that the MAB UCB is superior to single-tier systems. The MAB UCB can achieve execution time within a close margin of an oracle scheduler and also can fail in some extreme cases.