Big Data on AWS

COURSE OUTLINE:

Description

In this course, you will learn about cloud-based big data solutions such as Amazon Elastic MapReduce (EMR), Amazon Redshift, Amazon Kinesis, and the rest of the AWS big data platform. You will learn how to use Amazon EMR to process data using the broad ecosystem of Apache Hadoop tools like Hive and Hue. Additionally, you will learn how to create big data environments, work with Amazon DynamoDB, Amazon Redshift, and Amazon Kinesis, and leverage best practices to design big data environments for security and cost-effectiveness.

Audience

  • Individuals responsible for designing and implementing big data solutions, such as solutions architects and system operator administrators
  • Data scientists and data analysts interested in learning about big data solutions on AWS

Prerequisites

  • Basic familiarity with big data technologies, including Apache Hadoop and HDFS
  • Complete the Big Data Technology Fundamentals web-based training or have equivalent experience
  • Working knowledge of core AWS services and public cloud implementation
  • Basic understanding of data warehousing, relational database systems, and database design

Learning Objectives

  • Fit AWS solutions inside of a big data ecosystem
  • Leverage Apache Hadoop in the context of Amazon EMR
  • Components of an Amazon EMR cluster
  • Launch and configure an Amazon EMR cluster
  • Leverage common programming frameworks available for Amazon EMR including Hive, Pig, and Streaming
  • Use Hue to improve the ease-of-use of Amazon EMR
  • Use in-memory analytics with Spark and Spark SQL on Amazon EMR
  • Choose appropriate AWS data storage options
  • Benefits of using Amazon Kinesis for near real-time big data processing
  • Data warehousing and columnar database concepts
  • Leverage Amazon Redshift to efficiently store and analyze data
  • Manage costs and security for Amazon EMR and Amazon Redshift deployments
  • Options for ingesting, transferring, and compressing data
  • Use visualization software to depict data and queries
  • Orchestrate big data workflows using AWS Data Pipeline

1. Overview of Big Data

2. Data Ingestion, Transfer, and Compression

3. AWS Data Storage Options

4. Using DynamoDB with Amazon EMR

5. Using Kinesis for Near Real-Time Big Data Processing

6. Introduction to Apache Hadoop and Amazon EMR

7. Using Amazon Elastic MapReduce

8. The Hadoop Ecosystem

9. Using Hive for Advertising Analytics

10. Using Streaming for Life Sciences Analytics

11. Using Hue with Amazon EMR

12. Running Pig Scripts with Hue on Amazon EMR

13. Spark on Amazon EMR

14. Running Spark and Spark SQL Interactively on Amazon EMR

15. Using Spark and Spark SQL for In-Memory Analytics

16. Managing Amazon EMR Costs

17. Securing your Amazon EMR Deployments

18. Data Warehouses and Columnar Datastores

19. Introduction to Amazon Redshift

20. Optimizing Your Amazon Redshift Environment

21. The Big Data Ecosystem on AWS

22. Visualizing and Orchestrating Big Data

23. Using Tibco Spotfire to Visualize Big Data

 

Note: This is an emerging technology course. Course outline is subject to change as needed.