Тег: IT
![](https://slavlotski.com/pictures/thumbs/apache-airflow-1.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/directed-acyclic-graph.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/directed-cyclic-graph.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/main-definitions-airflow.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/web-server-apache-airflow.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/airflow-architecture.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/execution_date-explain.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/trigger-rules.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/sensors-in-dag.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/failed-sensors-apache-airflow.thumb@2x.jpg)
В 2014 году компания Airbnb стремительно развивалась, что привело к увеличению объема данных и усложнению рабочих процессов обработки данных
![](https://slavlotski.com/pictures/thumbs/apache-kafka.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-kafka-1.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-kafka-15.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-kafka-7.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-kafka-6.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-kafka-4.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-kafka-3.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-kafka-8.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-kafka-5.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-kafka-9.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/a52b72b1495d94797ab6f8642a09219f.thumb@2x.jpg)
Message broker — это тип построения архитектуры, при котором элементы системы «общаются» друг с другом с помощью посредника
![](https://slavlotski.com/pictures/thumbs/apache-spark.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-spark-1.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-spark-core.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-spark-4.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-spark-rm.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-spark-cluster-deploy.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/1692292839305.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-spark-3.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/image-2.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-spark-5.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-spark-6.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-spark-8.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-spark-9.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-spark-10.thumb@2x.jpg)
Apache Spark — это универсальный, высокопроизводительный, отказоустойчивый движок, написанный на Scala, для распределенной обработки данных
![](https://slavlotski.com/pictures/thumbs/apache-hadoop.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-hadoop-1.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-hadoop-4.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-hadoop-3.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-hadoop-5.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-hadoop-10.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-hadoop-6.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-hadoop-9.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-hadoop-8.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-hadoop-11.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-hadoop-12.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-hadoop-13.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/apache-hadoop-2.thumb@2x.jpg)
Из-за значительного роста обрабатываемых данных (терабайты, петабайты) действующие на тот момент системы уже эффективно не справлялись с обработкой такого потока данных
![](https://slavlotski.com/pictures/thumbs/relyacionnye-i-neryalyacionnye-bd.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/relyacionnye-i-neryalyacionnye-bd-1.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/relyacionnye-i-neryalyacionnye-bd-2.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/relyacionnye-i-neryalyacionnye-bd-3.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/relyacionnye-i-neryalyacionnye-bd-5.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/remote/youtube-IBzTDkYNB7I-cover.thumb@2x.jpg)
SQL подход — это семейство реляционных баз данных, основанное на отношениях (связях) таблиц друг с другом
![](https://slavlotski.com/pictures/thumbs/traditional-db-and-mpp-db-2.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/mpp-architecture.thumb@2x.jpg)
В данной статье разберу чем традиционные БД отличаются от MPP, в каких задачах достаточно иметь традиционную, а в каких MPP значительно лучше
![](https://slavlotski.com/pictures/thumbs/Kolesa-conf2023.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/kolesa-conf23.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/kolesa-conf23-1.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/kolesa-conf23-2.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/telegram-cloud-photo-size-2-5274177951128210823-y.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/remote/youtube-unibiCxOW2Y-cover.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/remote/youtube-G0tBWwdpUtU-cover.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/remote/youtube-nAPrLVq5MbE-cover.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/remote/youtube-b0YkT6VLAPA-cover.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/remote/youtube-8migJjRFD68-cover.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/telegram-cloud-photo-size-2-5353041930163965879-y.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/IvanStepanov_131.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/1A0A3597.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/1A0A3592.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/1A0A3772.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/1A0A3577.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/1A0A5069.thumb@2x.jpg)
![](https://slavlotski.com/pictures/thumbs/remote/youtube-xDdngNVCNc4-cover.thumb@2x.jpg)
07.10.2023 в Алмате состоялась ежегодная конференция Kolesa Conf’23. Kolesa Conf’23 — это масштабная конференция, объединяющая IT-сообщество Казахстана