Apache PIG & HIVE Training

• List the Three "V"s of Big Data

• List the Six Key Hadoop Data Types

• Describe Hadoop, YARN and Use Cases for Hadoop

• Describe Hadoop Ecosystem Tools and Frameworks

• Describe the Differences Between Relational Databases and Hadoop

• Describe What is New in Hadoop 2.x

• Describe the Hadoop Distributed File System (HDFS)

• Describe the Differences Between HDFS and an RDBMS

• Describe the Purpose of NameNodes and DataNodes

• List Common HDFS Commands

• Describe HDFS File Permissions

• List Options for Data Input

• Describe WebHDFS

• Describe the Purpose of Sqoop and Flume

• Describe How to Export to a Table

• Describe the Purpose of MapReduce

• Define Key/Value Pairs in MapReduce

• Describe the Map and Reduce Phases

• Describe Hadoop Streaming

• Starting an HDP Cluster

• Demonstration: Understanding Block Storage (Lab)

• Using HDFS Commands (Lab)

• Importing RDBMS Data into HDFS (Lab)

• Exporting HDFS Data to an RDBMS (Lab)

• Importing Log Data into HDFS Using Flume (Lab)

• Demonstration: Understanding MapReduce (Lab)

• Running a MapReduce Job (Lab)

• Describe the Purpose of Apache Pig

• Describe the Purpose of Pig Latin

• Demonstrate the Use of the Grunt Shell

• List Pig Latin Relation Names and Field Names

• List Pig Data Types

• Define a Schema

• Describe the Purpose of the GROUP Operator

• Describe Common Pig Operators ( ORDER BY, CASE, DISTINCT, PARALLEL, FLATTEN, FOREACH)

• Perform an Inner, Outer and Replicated Join

• Describe the Purpose of the DataFu Library

• Demonstration: Understanding Apache Pig (Lab)

• Getting Starting with Apache Pig (Lab)

• Exploring Data with Apache Pig (Lab)

• Splitting a Dataset (Lab)

• Joining Datasets with Apache Pig (Lab)

• Preparing Data for Apache Hive (Lab)

• Demonstration: Computing Page Rank (Lab)

• Analyzing Clickstream Data (Lab)

• Analyzing Stock Market Data Using Quantiles (Lab)

• Describe the Purpose of Apache Hive

• Describe the Differences Between Apache Hive and SQL

• Describe the Apache Hive Architecture

• Demonstrate How to Submit Hive Queries

• Describe How to Define Tables

• Describe How to Load Date Into Hive

• Define Hive Partitions, Buckets and Skew

• Describe How to Sort Data

• List Hive Join Strategies

• Describe the Purpose of HCatalog

• Describe the HCatalog Ecosystem

• Define a New Schema

• Demonstrate the Use of HCatLoader and HCatStorer with Apache Pig

• Perform a Multi-table/File Insert

• Describe the Purpose of Views

• Describe the Purpose of the OVER Clause

• Describe the Purpose of Windows

• List Hive Analytics Functions

• List Hive File Formats

• Describe the Purpose of Hive SerDe

• Understanding Hive Tables (Lab)

• Understanding Partition and Skew (Lab)

• Analyzing Big Data with Apache Hive (Lab)

• Demonstration: Computing NGrams (Lab)

• Joining Datasets in Apache Hive (Lab)

• Computing NGrams of Emails in Avro Format (Lab)

• Using HCatalog with Apache Pig (Lab)

• Describe the Purpose HDFS Federation

• Describe the Purpose of HDFS High Availability (HA)

• Describe the Purpose of the Quorum Journal Manager

• Demonstrate How to Configure Automatic Failover

• Describe the Purpose of YARN

• List the Components of YARN

• Describe the Lifecycle of a YARN Application

• Describe the Purpose of a Cluster View

• Describe the Purpose of Apache Slider

• Describe the Origin and Purpose of Apache Spark

• List Common Spark Use Cases

• Describe the Differences Between Apache Spark and MapReduce

• Demonstrate the Use of the Spark Shell

• Describe the Purpose of an Resilient Distributed Dateset (RDD)

• Demonstrate How to Load Data and Perform a Word Count

• Define Lazy Evaluation

• Describe How to Load Multiple Types of Data

• Demonstrate How to Perform SQL Queries

• Demonstrate How to Perform DataFrame Operations

• Describe the Purpose of the Optimization Engine

• Describe the Purpose of Apache Oozie

• Describe Apache Pig Actions

• Describe Apache Hive Actions

• Describe MapReduce Actions

• Describe How to Submit an Apache Oozie Workflow

• Define an Oozie Coordinator Job

• Advanced Apache Hive Programming (Lab)

• Running a YARN Application (Lab)

• Getting Started with Apache Spark (Lab)

• Exploring Apache Spark SQL (Lab)

• Defining an Apache Oozie Workflow (Lab)