Apache Spark Course

Spark Architecture in 6 minutes

#apachespark #spark #sparkarchitecture
📓 Application
📓 Driver
📓 Executor
📓 Partition
📓 Job
📓 Stage
📓 Tasks
📓 Slots
📓 Lazy evaluation
📓 Narrow and wide Transformations
📓 Actions

PySpark Introduction in 7 minutes

#pyspark #apachespark

📗 Data Frame Creation
📗 Eager evaluation
📗 View the data frames and manipulations
📗 SQL with pyspark
📗 Group By

🔋 GitHub: https://github.com/ambarishg/pyspark

Spark SQL Case Study with Used Car Dataset [ 1.4 GB ] in 10 minutes

⭐ Schema Enforcement
⭐ COUNT
⭐ GROUP BY Single variable, Multiple variables
⭐ Average, Standard Deviation, Maximum, Minimum
⭐ ROLLUP and CUBE
⭐ Dense Rank without and with Partition

🔋 GitHub: https://github.com/ambarishg/pyspark/tree/master/02-SparkSQL

Lakehouse Architecture in 9 minutes

#apachespark #spark #sparkarchitecture

✔️Evolution of data warehouses to Lakehouse
✔️SQL Performance Techniques in Lakehouse
✔️ Brief about Delta lake

Delta Tables Deep Dive in 14 minutes [ Case Study approach - Cricket Test Match ]

#apachespark #spark #deltatable