![SelfReflex](/img/default-banner.jpg)
- 52
- 143 230
SelfReflex
India
Приєднався 5 лют 2016
SelfReflex is the leading provider of free training resources for Hadoop, Apache Spark, MapReduce and big data. Our aim is to help as many learners as possible without charging even one penny. We are a bunch of passionate geeks who love to code & coach. Our decades of experience in IT industry (primarily in big data world) has enabled us to connect the dots between the theory & practice. We know the underground pumping & the nitty gritty of real world systems. You name few exceptions/errors and chances are - we have dealt with them! Subscribe to our youtube channel and start learning!
Batch vs Stream processing | Spark Interview Questions
What is the difference between batch processing and stream processing? in context of spark streaming interviews. Whats real time processing?
Переглядів: 203
Відео
Apache Spark Tutorial: SQL like Functions on Datasets - select, filter, groupBy - hands on
Переглядів 78Рік тому
Welcome to the immersive 13th session of our comprehensive Apache Spark tutorial series! In this session, we dive deep into the world of hands-on exercises, focusing on SQL-like functions on Datasets. Get ready to master the art of data manipulation in Apache Spark! In this session, we'll guide you through a series of practical exercises that harness the power of SQL-like functions on Datasets....
Apache Spark Tutorial: Hands-on Spark SQL, Datasets, and DataFrames: Dataset vs Dataframe
Переглядів 96Рік тому
Welcome to the exhilarating 12th session of our comprehensive Apache Spark tutorial series! In this session, we dive straight into the world of hands-on exercises with Spark SQL, Datasets, and DataFrames, taking you on an immersive journey of practical structured data processing in Apache Spark. In this session, we roll up our sleeves and embark on a series of hands-on exercises designed to sol...
Apache Spark Tutorial: Introduction To Spark SQL, Datasets, and DataFrames
Переглядів 108Рік тому
Welcome to the captivating 11th video of our comprehensive Apache Spark tutorial series! In this session, we dive into the world of Spark SQL, Datasets, and DataFrames, where we unlock the full potential of structured data processing in Apache Spark. In this session, we take you on a journey through the rich capabilities of Spark SQL, the module that enables SQL-like querying and processing of ...
Apache Spark Tutorial: Mastering Jobs, Stages, Tasks, and Spark's logical execution plan
Переглядів 667Рік тому
Welcome to the captivating tenth session of our comprehensive Apache Spark tutorial series! In this session, we delve into the core concepts of Spark's data processing engine, where we demystify the intricate relationship between jobs, stages, tasks, and the all-important shuffle sort operation. In this session, we provide a comprehensive overview of how Spark organizes and executes data proces...
Apache Spark Tutorial: Deep Dive into Apache Spark Runtime Architecture on YARN - Spark Internals
Переглядів 116Рік тому
Welcome to the enlightening ninth session of our comprehensive Apache Spark tutorial series! In this session, we embark on a fascinating exploration of the Spark runtime architecture on YARN (Yet Another Resource Negotiator). Join us as we peel back the layers and unravel the inner workings of Spark's execution framework in a YARN cluster environment. In this session, we provide an in-depth ove...
Apache Spark Tutorial: Broadcast Variables Hands-On Exercise: Real-Life Use Case From Telecom World
Переглядів 83Рік тому
Welcome to the highly practical eighth session of our comprehensive Apache Spark tutorial series! In this session, we delve into the powerful world of broadcast variables and guide you through a hands-on exercise using a real-life use case from the telecom industry. Join us as we explore how broadcast variables can optimize data sharing in large-scale telecom data processing and analytics. In t...
Apache Spark Tutorial: Deep Dive into Broadcast Variables in Apache Spark - Unveiling the Internals
Переглядів 75Рік тому
Welcome to the captivating seventh session of our comprehensive Apache Spark tutorial series! In this session, we embark on an exciting journey into the inner workings of broadcast variables in Spark. Join us as we unravel the mysteries behind broadcast variables, exploring how they optimize data sharing and improve performance in distributed computations. We dive into the internals, shedding l...
Apache Spark Tutorial: map vs flatmap
Переглядів 126Рік тому
Welcome to the highly informative sixth session of our comprehensive Apache Spark tutorial series! In this session, we delve into the essential concepts of the Map and FlatMap operations in Spark, highlighting the key differences between them. Join us as we unravel the nuances of Map and FlatMap and explore when to use each operation effectively. By the end of this session, you'll have a solid ...
Apache Spark Tutorial: RDD Filter Operation Explained with Hands-on Exercise for Data Filtering
Переглядів 126Рік тому
Welcome to the highly anticipated fifth session of our comprehensive Apache Spark tutorial series! In this exciting session, we deep-dive into the RDD filter operation and its powerful capabilities for data filtering in Spark. Join us as we demystify the filter operation through a hands-on exercise that will sharpen your skills in data filtering and transformation. We'll guide you step-by-step ...
Apache Spark Tutorial: Exploring Key-Value RDDs with Hands-on Exercise for Big Data Processing
Переглядів 116Рік тому
Welcome to the highly anticipated fourth session of our comprehensive Apache Spark tutorial series! In this dynamic session, we delve into the intricacies of Key-Value RDDs (Resilient Distributed Datasets) and demonstrate their immense value in advanced big data processing. Join us as we uncover the power of Key-Value RDDs through hands-on exercises and practical examples. Through this immersiv...
How to decide number of executors | Apache Spark Interview Questions
Переглядів 116Рік тому
How to decide number of executors for a spark job? AN important interview question for apache spark. watch to find out the answer.
When does Apache Spark compute RDD Lineage | Apache Spark Interview Questions with answers
Переглядів 34Рік тому
When does Apache Spark compute RDD Lineage? watch to figure out how to answer this interview question on apache spark.
Is Spark better than Hadoop? | Apache Spark Interview Questions | With Explanation
Переглядів 47Рік тому
Apache spark interview question. Is spark better than hadoop? watch to know the answer. If you think you know the answer, you might be wrong. If you think the question is wrong, watch to find out.
Apache Spark Tutorial: Hands-on Exercise with RDDs for Big Data Processing | Learning RDDs
Переглядів 179Рік тому
Welcome back to our engaging Apache Spark tutorial series! In this exciting third session, we roll up our sleeves and dive into a hands-on exercise with RDDs (Resilient Distributed Datasets) to unleash the full potential of Spark for big data processing. Join us as we guide you through a step-by-step practical exercise, where you'll gain invaluable experience in manipulating RDDs, applying tran...
Apache Spark Tutorial: Mastering RDDs (Resilient Distributed Datasets) for Big Data Processing
Переглядів 231Рік тому
Apache Spark Tutorial: Mastering RDDs (Resilient Distributed Datasets) for Big Data Processing
Apache Spark Tutorial: Introduction and Basics of Big Data Processing
Переглядів 594Рік тому
Apache Spark Tutorial: Introduction and Basics of Big Data Processing
Hadoop interview questions Explain map reduce paradigm with example
Переглядів 1,2 тис.7 років тому
Hadoop interview questions Explain map reduce paradigm with example
Hadoop interview questions map reduce vs traditional data processing
Переглядів 8137 років тому
Hadoop interview questions map reduce vs traditional data processing
Hadoop interview questions what is map reduce
Переглядів 1,2 тис.7 років тому
Hadoop interview questions what is map reduce
Hadoop interview questions hadoop 1 0 vs hadoop 2 0
Переглядів 9657 років тому
Hadoop interview questions hadoop 1 0 vs hadoop 2 0
Hadoop interview questions reason for write failures
Переглядів 1 тис.7 років тому
Hadoop interview questions reason for write failures
Hadoop interview questions secondary vs standby namenode
Переглядів 1,5 тис.7 років тому
Hadoop interview questions secondary vs standby namenode
Hadoop interview questions hdfs node failures
Переглядів 1,3 тис.7 років тому
Hadoop interview questions hdfs node failures
hadoop interview questions what is rack awareness
Переглядів 6 тис.7 років тому
hadoop interview questions what is rack awareness
Great explanation !
best explanation not only on youtube but the entire internet .............Thank you so much
You are welcome!
after watching lots of videos and getting more confused this video showed up and unexpectedly turned out to be a life saver hats off to you sir
do you have a simple code for the traditional way or another more explicit video?
This 45 minutes lecture is a master piece..!!! Nothing else is even close to this one.
Thanks for the amazing explanations Omar from Egypt :)
Glad it was helpful!
@@Selfreflex It was helpful indeed. I got an A+ in my final exams in big data 🤓
@@MrPrince keep it up! Looking forward to serve more content for the learners like you!
Hi can you share your contact....
One slave node can have more than one container right?
Consider a retail shop with 10 employees where each employees gets a monthly bonous of he number of sales select * from employee distributed by employee_id sorted by sales_amount If the above command executed, how many reducer will be used ?? A)Canot be determined B)1.0 C)no reducer will be used D)10 Kindly ans my questions please
Before coming to this video, I wasted 2 days searching many websites and videos for understanding YARN..! I hesitated to take video b/z of time but I AM LUCKY ALL MY DOUBTS GOT CLEARED ON YARN..! THANKS.. 👍🙏
Glad I could help!
thank you so much its a clear explanation
Nicely explained sir
Thanks
sir can you share this ppt with us
Awesome sir. Thank you for giving this. Your website is not working .plz upload many videos like this....
U made it easy
Thanks
According to your explanation it looks like no. of mapper is equal to no. of block size ...but generally no. of mapper is equal to no. of input split..pls let me know if i am wrong 🤔
greate job
Thanks
Well explained. Great job. Thank you.
Secondary namenode is standby namenode
before coming here i searched all the blogs about yarn but didnt satisfied but this 45min cleared all the concepts even how actual data process i understood.Thank you sir,awesome explanation,VERY USEFULL
Very nicely explained the concept of split size.
Totally worth it, a clean explanation!
The best YARN explanation . Thank you !
Best YARN Lecture. Sir, Can you please explain? How to configure capacity-scheduler for queue for submit multiple spark application at same time? Or any other suggestion to run multiple application on Hadoop yarn cluster. Please
The best YARN lecture ever! 45 minutes are completely worth it!
Another question I have is what is the resource manager died but the data was processed and manipulated and is no longer in its original state?
Difference between yarn and map reduce?
map reduce is single point of failure yarn is not single point
superb explaination
Great explanation, make videos for other services, hadoop admin etc, issues as well
Oulryte - goodbye
Thanks professor ..Your throat deserves some rest yes !!Nice explanations
How many container could be launch in one node?
First of all your fsimage is not present on the shared storage (i.e Journal Nodes), it only contains edit logs which records changes in HDFS fs. Both the active as well as standby NNs store the fsimage on the local disk hosting the daemons.
Awesouy. Right video to learn Yarn
Awesome
Thanks for sharing your knowledge
Great video. Thank you so much
please try to explained Hadoop Resource manager High availability in deep.
When ApM no requested container at the node where data is stored, but RM not able to find any , ApM is given container and has to process data over remote access. In case of network failures in remote access, who and what actions are taken?
one of the best I have seen for yarn understanding..(Y)
much better than edureka!
Very nice
In 1st scenario, if 2nd block is present in different node. Then how the 3rd line is read by the record reader?
Very very good Explanation
Thanks
@@Selfreflex Can you please upload videos on Spark 2
False !!
Crystal Clear explanation, in Simple English . Thumbs up to you. Great tutorial for HDFS. I understood a lot from this video. Thank you so much.
@44:13 - Kumar - can't stop laughing
how the recordreader will know the starting of data if the data is going in block2?
Excellent explanation...simple