Distributed Data Management (ST 2021) - tele-TASK

By Dr. Thorsten Papenbrock

Listen to a podcast, please open Podcast Republic app. Available on Google Play Store and Apple App Store.


Category: Courses

Open in Apple Podcasts


Open RSS feed


Open Website


Rate for this podcast

Subscribers: 1
Reviews: 0
Episodes: 26

Description

The free lunch is over! Computer systems up until the turn of the century became constantly faster without any particular effort simply because the hardware they were running on increased its clock speed with every new release. This trend has changed and today's CPUs stall at around 3 GHz. The size of modern computer systems in terms of contained transistors (cores in CPUs/GPUs, CPUs/GPUs in compute nodes, compute nodes in clusters), however, still increases constantly. This caused a paradigm shift in writing software: instead of optimizing code for a single thread, applications now need to solve their given tasks in parallel in order to expect noticeable performance gains. Distributed computing, i.e., the distribution of work on (potentially) physically isolated compute nodes is the most extreme method of parallelization. Big data analytics and management are a multi-million dollar markets that grow constantly! The ability to control and utilize large amounts of data is the most valuable ability of today's computer systems. Because data volumes grow so rapidly and with them the complexity of questions they should answer, data analytics, i.e., the ability of extracting any kind of information from the data becomes increasingly difficult. As data analytics systems cannot hope for their hardware getting any faster to cope with performance problems, they need to embrace new software trends that let their performance scale with the still increasing number of processing elements. In this lecture, we take a look at various technologies involved in building distributed, data-intensive systems. We start by discussing fundamental concepts in distributed computing, such das data models, encoding formats, messaging, data replication and partitioning, fault tollerance, and batch- and stream processing. In between, we consider different practical systems from the Big Data Landscape, such as Akka and Spark. In the end, we concentrate on data management aspects, such as distributed database management system architectures and distributed query optimization.

Episode Date
Lecture Summary
Jul 21, 2021
Federated DBMSS
Jul 19, 2021
Stream Processing - Databases and Streams
Jul 12, 2021
Stream Processing
Jul 07, 2021
Exercise 1 Evaluation
Jul 05, 2021
Spark Batch Processing (2)
Jun 30, 2021
Spark Batch Processing
Jun 28, 2021
Batch Processing 2 - Distributed File Systems and MapReduce
Jun 21, 2021
Batch Processing
Jun 16, 2021
Transactions
Jun 14, 2021
Consistency and Consensus
Jun 09, 2021
Distributed Systems
Jun 07, 2021
Partitioning & Distributed Systems
Jun 02, 2021
Replication & Partitioning
May 31, 2021
Replication
May 26, 2021
Storage and Retrieval
May 19, 2021
Data Models and Query Languages
May 17, 2021
Akka Actor Programming 3 - Patterns
May 12, 2021
Akka Actor Programming 2
May 05, 2021
Akka Actor Programming
May 03, 2021
Communication: Service-oriented & Database-oriented Middleware
Apr 28, 2021
Communication: Message-oriented Middleware
Apr 26, 2021
Communication
Apr 21, 2021
Encoding
Apr 19, 2021
Foundations
Apr 14, 2021
Introduction
Apr 12, 2021