Distributed Data Management (WT 2018/19) - tele-TASK

By Prof. Dr. Felix Naumann, Dr. Thorsten Papenbrock

Listen to a podcast, please open Podcast Republic app. Available on Google Play Store and Apple App Store.


Category: Courses

Open in Apple Podcasts


Open RSS feed


Open Website


Rate for this podcast

Subscribers: 0
Reviews: 0
Episodes: 26

Description

The free lunch is over! Computer systems up until the turn of the century became constantly faster without any particular effort simply because the hardware they were running on increased its clock speed with every new release. This trend has changed and today's CPUs stall at around 3 GHz. The size of modern computer systems in terms of contained transistors (cores in CPUs/GPUs, CPUs/GPUs in compute nodes, compute nodes in clusters), however, still increases constantly. This caused a paradigm shift in writing software: instead of optimizing code for a single thread, applications now need to solve their given tasks in parallel in order to expect noticeable performance gains. Distributed computing, i.e., the distribution of work on (potentially) physically isolated compute nodes is the most extreme method of parallelization. Big Data Analytics is a multi-million dollar market that grows constantly! Data and the ability to control and use it is the most valuable ability of today's computer systems. Because data volumes grow so rapidly and with them the complexity of questions they should answer, data analytics, i.e., the ability of extracting any kind of information from the data becomes increasingly difficult. As data analytics systems cannot hope for their hardware getting any faster to cope with performance problems, they need to embrace new software trends that let their performance scale with the still increasing number of processing elements. In this lecture, we take a look a various technologies involved in building distributed, data-intensive systems. We discuss theoretical concepts (data models, encoding, replication, ...) as well as some of their practical implementations (Akka, MapReduce, Spark, ...). Since workload distribution is a concept which is useful for many applications, we focus in particular on data analytics.

Episode Date
Lecture Summary
Feb 05, 2019
Distributed Query Optimization (1)
Jan 22, 2019
Distributed Query Optimization (2)
Jan 22, 2019
Processing Streams
Jan 15, 2019
Stream Processing
Jan 14, 2019
Transactions
Jan 08, 2019
Consistency and Consensus
Jan 07, 2019
Distributed Systems
Dec 18, 2018
Spark - Hands On
Dec 17, 2018
Apache Spark
Dec 11, 2018
Beyond MapReduce
Dec 10, 2018
Distributed File Systems and MapReduce
Dec 04, 2018
Batch Processing
Dec 03, 2018
Partitioning
Nov 27, 2018
Replication
Nov 26, 2018
Storage and Retrieval
Nov 20, 2018
Data Models and Query Languages
Nov 13, 2018
Patterns
Nov 12, 2018
Akka Actor-Programming Part 2
Nov 06, 2018
Akka Actor-Programming Hands-on
Nov 05, 2018
Models of Dataflow
Oct 30, 2018
Encoding and Evolution
Oct 29, 2018
Data Warehouses
Oct 23, 2018
Distributed DBMS
Oct 22, 2018
Foundations
Oct 16, 2018
Introduction
Oct 15, 2018