Tech & IT
Business
Coding & Developer
Finance & Accounting
Academics
Office Applications
Art & Design
Marketing
Health & Wellness
Sounds & Music
Lifestyle
Photography
Get Unlimited Learning Access
$29
5 days left at this price!
30-Day Money-Back Guarantee
This plan includes
- Instant access to 11,000+ online courses
- Play & Pause Course Videos
HD Video Recorded Lectures- Learn on Mobile/PC/Tablet
- Quizzes and Real Projects
- Lifetime Course Certificate
- Instructor Chat Support
- Cancel Plan Anytime
Subscribe to Learnfly’s top courses
Get this course, plus 11,000+ of our top-rated courses for one year with Go Annually Plan.
5 days left at this price!
30-Day Money-Back Guarantee
This plan includes
- Instant access to 11,000+ online courses
- Play & Pause Course Videos
- HD Video Recorded Lectures
- Learn on Mobile/PC/Tablet
- Quizzes and Real Projects
- Lifetime Course Certificate
- Instructor Chat Support
- Cancel Plan Anytime
$29
What you'll learn?
- Hadoop distributed File system and commands. Lifecycle of sqoop command. Sqoop import command to migrate data from Mysql to HDFS. Sqoop import command to migrate data from Mysql to Hive. Working with various file formats, compressions, file delimeter,where clause and queries while importing the data. Understand split-by and boundary queries. Use incremental mode to migrate the data from Mysql to HDFS. Using sqoop export, migrate data from HDFS to Mysql. Using sqoop export, migrate data from Hive to Mysql. Understand Flume Architecture. Using flume, Ingest data from Twitter and save to HDFS. Using flume, Ingest data from netcat and save to HDFS. Using flume, Ingest data from exec and show on console. Flume Interceptors.
Course Overview
In this course, you will start by learning what is hadoop distributed file system and most common hadoop commands required to work with Hadoop File system.
Then you will be introduced to Sqoop Import
Understand lifecycle of sqoop command.
Use sqoop import command to migrate data from Mysql to HDFS.
Use sqoop import command to migrate data from Mysql to Hive.
Use various file formats, compressions, file delimeter,where clause and queries while importing the data.
Understand split-by and boundary queries.
Use incremental mode to migrate the data from Mysql to HDFS.
Further, you will learn Sqoop Export to migrate data.
What is sqoop export
Using sqoop export, migrate data from HDFS to Mysql.
Using sqoop export, migrate data from Hive to Mysql.
Further, you will learn about Apache Flume
Understand Flume Architecture.
Using flume, Ingest data from Twitter and save to HDFS.
Using flume, Ingest data from netcat and save to HDFS.
Using flume, Ingest data from exec and show on console.
Describe flume interceptors and see examples of using interceptors.
Flume multiple agents
Flume Consolidation.
Finally, we will start with our last section about Apache Hive
Hive Intro
External & Managed Tables
Working with Different Files - Parquet,Avro
Compressions
Hive Analysis
Hive String Functions
Hive Date Functions
Partitioning
Bucketing
Pre-requisites
- Cloudera vm installation if you want to run examples.
Target Audience
- Who want to learn sqoop and flume or who are preparing for CCA and HDPCD certifications
Curriculum 70 Lectures 05:47:17
-
Section 1 : Hadoop Introduction
- Lecture 2 :
- Mapreduce Yarn
- Lecture 3 :
- Cloudera VM Machine Setup Guide
-
Section 2 : Apache Sqoop Import
- Lecture 1 :
- Managing Target Directories
- Lecture 2 :
- Working with Different File Formats
- Lecture 3 :
- Working with Different Compressions
- Lecture 4 :
- Conditional Imports
- Lecture 5 :
- Split-by and Boundary Queries
- Lecture 6 :
- Fiield delimeters
- Lecture 7 :
- Incremental Appends
- Lecture 8 :
- Sqoop Hive Import
- Lecture 9 :
- Sqoop List Tables/Database
- Lecture 10 :
- Sqoop Import Practice 1
- Lecture 11 :
- Sqoop Import Practice 2
- Lecture 12 :
- Sqoop Import Practice 3
- Lecture 13 :
- Sqoop intro
-
Section 3 : Sqoop Export
- Lecture 1 :
- Export from Hdfs to Mysql
- Lecture 2 :
- Export from Hive to Mysql
-
Section 4 : Apache Flume
- Lecture 1 :
- Flume Introduction & Architecture
- Lecture 2 :
- Exec Source and Logger Sink
- Lecture 3 :
- Moving data from Twitter to HDFS
- Lecture 4 :
- Moving data from NetCat to HDFS
- Lecture 5 :
- Flume Interceptors
- Lecture 6 :
- Flume Interceptor Example
- Lecture 7 :
- Flume Multi-Agent Flow
- Lecture 8 :
- Fume Consolidation
-
Section 5 : Apache Hive
- Lecture 1 :
- Hive Introduction
- Lecture 2 :
- Hive Database
- Lecture 3 :
- Hive Managed Tables
- Lecture 4 :
- Hive External Tables
- Lecture 5 :
- Hive Inserts
- Lecture 6 :
- Hive Analytics
- Lecture 7 :
- Hive Parquet
- Lecture 8 :
- Parquet Snappy
- Lecture 9 :
- Working with Fixed File Format
- Lecture 10 :
- Alter Command
- Lecture 11 :
- Hive String Functions
- Lecture 12 :
- Hive Partitioning
- Lecture 13 :
- Hive Bucketing
- Lecture 14 :
- Hive Date Functions
-
Section 6 : Spark Introduction
- Lecture 1 :
- Spark Intro
- Lecture 2 :
- Resilient Distributed Dataset
- Lecture 3 :
- Cluster Overview
- Lecture 4 :
- DAG Stages
-
Section 7 : Spark Transformation Actions
- Lecture 1 :
- Map
- Lecture 2 :
- Filter Intersect
- Lecture 3 :
- Union Distinct
- Lecture 4 :
- Group By Key
- Lecture 5 :
- Reduce By Key
- Lecture 6 :
- Sort By Key
- Lecture 7 :
- Map Partition
- Lecture 8 :
- Coleasce Repartition
- Lecture 9 :
- Join
- Lecture 10 :
- Spark Actions
-
Section 8 : Spark RDD Practice
- Lecture 1 :
- Scala Tuples
- Lecture 2 :
- Extract Error
- Lecture 3 :
- Word Frequency
- Lecture 4 :
- Customer Orders
- Lecture 5 :
- City Population
- Lecture 6 :
- Movie Rating
-
Section 9 : Spark Dataframe SparkSQL
- Lecture 1 :
- Dataframe Intro
- Lecture 2 :
- Working With Json
- Lecture 3 :
- Working With Parquet
- Lecture 4 :
- Working With CSV
- Lecture 5 :
- Working With Avro XML
- Lecture 6 :
- Compressions
- Lecture 7 :
- Dataframe API 1
- Lecture 8 :
- Dataframe API 2
- Lecture 9 :
- Spark SQL
- Lecture 10 :
- Hive Context
Our learners work at
Frequently Asked Questions
-
How do i access the course after purchase?
It's simple. When you sign up, you'll immediately have unlimited viewing of thousands of expert courses, paths to guide your learning, tools to measure your skills and hands-on resources like exercise files. There’s no limit on what you can learn and you can cancel at any time. -
Are these video based online self-learning courses?
Yes. All of the courses comes with online video based lectures created by certified instructors. Instructors have crafted these courses with a blend of high quality interactive videos, lectures, quizzes & real world projects to give you an indepth knowledge about the topic. -
Can i play & pause the course as per my convenience?
Yes absolutely & thats one of the advantage of self-paced courses. You can anytime pause or resume the course & come back & forth from one lecture to another lecture, play the videos mulitple times & so on. -
How do i contact the instructor for any doubts or questions?
Most of these courses have general questions & answers already covered within the course lectures. However, if you need any further help from the instructor, you can use the inbuilt Chat with Instructor option to send a message to an instructor & they will reply you within 24 hours. You can ask as many questions as you want. -
Do i need a pc to access the course or can i do it on mobile & tablet as well?
Brilliant question? Isn't it? You can access the courses on any device like PC, Mobile, Tablet & even on a smart tv. For mobile & a tablet you can download the Learnfly android or an iOS app. If mobile app is not available in your country, you can access the course directly by visting our website, its fully mobile friendly. -
Do i get any certificate for the courses?
Yes. Once you complete any course on our platform along with provided assessments by the instructor, you will be eligble to get certificate of course completion.
-
For how long can i access my course on the platform?
You require an active subscription to access courses on our platform. If your subscription is active, you can access any course on our platform with no restrictions. -
Is there any free trial?
Currently, we do not offer any free trial. -
Can i cancel anytime?
Yes, you can cancel your subscription at any time. Your subscription will auto-renew until you cancel, but why would you want to?
Instructor
363090 Course Views
3 Courses
I am a big data professional with 11 years of industry experience in different technologies and domains. I have keen interest in providing training in new technologies. I have received CCA175 Hadoop and Spark developer certification and AWS solution architect certification. I love guiding people and helping them achieve new goals.
View More...
