• Self Paced: 11 Sessions
  • |
  • CloudxLab™: 90 days - 24x7, Global
  • |
  • Project: 2 wks
Self Paced Classes
Full Access To CloudxLab™ - 90x24 Hrs
Training By Industry Experts
Real Time Project
Earn Certificate In Big Data with Apache Spark
24x7 Online Expert Help
Python Basics
Kafka Basics
Course Query
Self Paced Classes
Starts Time Details Trainer Price
Self Paced Anytime
  • Self Paced Classes
  • Total 11 Sessions
Sandeep Giri

(20% off - Early Bird) (Incl. Taxes)
Upcoming Live Courses

  Big Data with Hadoop and Spark - Starts on 14th January, 2017


As the data has grown bigger, more complex requirements or rate at which you are processing data has higher, the traditional tools are no longer able to handle the data because a single computer does not suffice because of IO, CPU & RAM limitaton.

That is when the new generation of tools which run on multiple computers are required. And Apache spark is probably the fastest and most efficient amongst all other distributed computing tools.

This course will take you from the very basics to an advanced level in Big Data Analysis and Streaming processing using Apache Spark. It will be a very hands on Session.

We will start with the basics of Big Data and understand the architecture of Apache Spark and would solve problems using python on Spark. Then we will cover Spark SQL and basics of Machine Learning and solve few classic problems using MLLib.

Then we will understand how to process the fast moving data. Or in other words, we will do the stream processing using Spark.

At the end of this course, you should be able to do be able handle all three kinds of processing of Big Data:

  1. Volume (Spark SQL)
  2. Velocity (Spark Streaming)
  3. Variety (MLLib)

Why learn Big Data with Apache Spark?

Almost every organisation has humongous data that has to be analysed for business growth/increase sales or to improve customer service. So Data Analysis is the hot trend in every industry.

The other way to measure the demand for Data Analysts is to look at the number of jobs being posted around the world on these technologies.


Our classes are conducted live online by our instructors via webinar or hangout. These are not pre-recorded classes. The instructor delivers the class using presentations, collaborative drawing tools, screenshares. All attendees are usually muted during the class. However, they can ask questions in the webinar or hangout chat windows. The instructor answers any questions asked immediately after explaining a concept. The instructor also asks questions during the sessions to ensure maximum student engagement.

Every class is recorded, complete with the screen and the audio, and uploaded to the Learning Management System which is accessible to our attendees for life.

At the end of each session, assignments are provided which the attendees have to submit in the LMS (Learning Management System). The assignments are continuously reviewed by our instructors and teaching assistants. In case we conclude that an attendee requires extra detailing, we schedule extra one-on-one sessions with that attendee.

What makes Big Data with Apache Spark course unique?

  • Interactive Classes: More Questions. Less Lectures.
  • Simple explanations to complex topics by industry experts
  • Hands on workshops and real time projects.
  • Quizzes & Assignments
  • Certificate of Course at the end of course
  • A real time project involving Big Data with Apache Spark
  • Lifetime access to course content
  • CloudxLab™ - Access to the cloud infrastructure if learners don't wish to install Apache Spark on their computers

What are the prerequisites to join Big Data with Apache Spark course?

To be able to take maximum benefit out of this course, you should have knowledge of the following:

  1. Basics Of SQL. You should know the basics of SQL and databases.
  2. A know-how of the basics of programming.We will be providing video classes covering the basics of Python. What is expected of the attendee is the ability to create a directory and see whats inside a file from the command line, and an understanding of 'loops' in any programming language.

In addition, the attendee should have the following hardware infrastructure:

  • A good internet connection. An internet speed of 2mbps is good enough.
  • Access to a computer. Since it is an online course, you would have to install webinar or hangout on your computer.
  • Nice To Have: A power backup for your router as well as computer.
  • Nice To Have: A good quality headphones.

What kind of project / real time experience?

After all sessions are over, we ask for the student's preference for a project. We form teams of 3-4 members and based on their interests we assign a project to each team. A project is usually of three weeks duration. If a team has an idea it wants to work on as a project, we screen the idea and the team can work on it, or we assign a project from the industry. Since it is not possible to provide real data from the industry, we provide data anonymously for projects. We continuously support and guide the teams during projects by conducting regular scheduled meetings and also provide individual assistance.

The projects assigned can also be based on public databases. There are various datasets available for free that can be found on any of the following websites:

A few examples of projects are as follows:

  • Understanding the trends and patterns in BitCoin transaction graphs by qualitative analysis. BitCoin is a virtual currency. The way a coin is mined is based on transaction logs. BitCoin transaction logs keep growing almost every mili second, and therefore, processing these transaction logs is a real challenge.
  • Understanding the correlation between the temperature of various cities and the stock market.
  • Processing Apache Log for ERRORs. Preparing web analytics based on apache weblogs:
    • Which services are slow
    • Which services have a high number of users
    • What is the failure rate of each service
  • Preparing recommendations based on the apache logs.
  • Using social media to compare a brand's marketing campaigns. The testing is basically done using sentiment analysis.

Feedback from our alumni

Parveen Kumar

Ranjit Sahu

Associate Manager Technology at Thomson Reuters
LinkedIn Profile
Ratings: (5.0/5.0)
Review: Just completed the course "Big data & Hadoop" from KnowBigData. This is one of the best online course i have ever had. The instructor is amazing and his knowledge on the subject is excellent. The best about the course is, it is not one of those text-book course full of theories rather it is based on practical problems and how we can implement big data concepts to fix those.
Parveen Kumar

Parveen Kumar

VP - Engineering at CommonFloor
LinkedIn Profile
Ratings: (5.0/5.0)
Review: KnowBigData's "Big data & Hadoop" is one of the best courses I have attended online. Not only the instructor knows the concept extremely well but also very passionate about explaining difficult concept in simple way. The quiz is also very useful after some sessions to revise the learnings.
Gunjan Narulkar

Gunjan Narulkar

Data Scientist at Data Semantics
LinkedIn Profile
Ratings: (5.0/5.0)
Review: Just finished a course on Hadoop basics offered by KnowBigData.. An awesome experience in terms of exposure and learning to the full stack of technology.. but more importantly, to be in discussion with some one as experienced and brilliant and yet so grounded as Sandeep was far more enriching then anything else! I highly recommend this course for a holistic learning experience.
Soma Pandey

Soma Pandey

Consultant - Smart Grid Communications at Essel Vidyut Vitaran Nigam
PhD, Wireless Mesh Networks
LinkedIn Profile
Review: I attended Big Data classes and although I am from wireless communication area, still I was not only able to 'know big data' but also became well versed with it. The course is meticulously designed so as not to leave out any major topic on Big Data and its tools. Sandeep's method of teaching is excellent. Every question he answers with great patience and respect. I will strongly recommend this course to anyone who wishes to take up a career in this field. Other wise also people from any field who wish to diversify in this area must definitely take up this course.
Savita Singh

Savita Singh

Director Engineering, Target Technology Services
LinkedIn Profile
Ratings: (5.0/5.0)
Review: Joined the Hadoop class from Know BIG DATA 5 wks back and its been a motivating experience. Last I coded was 20yrs back and but thanks to the instructor led training - I am executing Pig Latin and Hive commands to solve data problems and look forward to soon be able to complete small projects all by myself. Sandeep has been a great instructor, very very patient, always ready to put in extra time to clarify doubts and work at your pace and schedule.
Dr. Makhan Virdi

Dr. Makhan Virdi

Researcher, NASA - DAAC
LinkedIn Profile
Ratings: (5.0/5.0)

Big Data with Apache Spark: This is not a typical (online) classroom course. It is not just a series of videos with one way flow of information. Instead, it is a highly interactive setting where the instructor shares insightful details when any question/doubt is raised during the lecture. Sandeep passionately teaches complicated concepts in easy to understand language, supported with good analogies and effective examples. The course is well structured, covering the concepts of Big Data in width and depth. I am currently half-way through the course and I am already working on translating the concepts learned in the class to real world problems.

Hari Madhav Purwar

Hari Madhav Purwar

Core Java Developer
LinkedIn Profile
Ratings: (5.0/5.0)

Review: I have been having difficulty understanding, concepts behind big data technology, being questioned around why we need big data, when we need it ?. Answers to these basic questions and then understanding different tools to handle such cases. Sandeep methodology, start from base and build learning on that base, helps in architect the problem towards right solutions.
Thanks Sandeep for such a wonderful sessions :)

Shashank Sharma

Shashank Sharma

Sr Oracle DBA at SCDL
LinkedIn Profile
Ratings: (5.0/5.0)
Review: I attended the knowbigdata classes.The course is very interesting and sandeep sir's way of teaching is excellent.He answers the questions very well and explain it in briefly to understand.And overall the environment in class was excellent.I am satisfied with Hadoop class.

See More Reviews at FaceBook Page.

Big Data with Apache Spark Introduction Video

Big Data with Apache Spark Course Curriculum

  1. What Is Apache Spark?
  2. A Unified Stack
  3. Who Uses Spark, and for What?
  4. A Brief History of Spark
  5. Spark Versions and Releases
  6. Storage Layers for Spark
  1. Downloading Spark
  2. Introduction to Spark’s Python and Scala Shells
  3. Introduction to Core Spark Concepts
  4. Standalone Applications
  5. Conclusion
  1. RDD Basics
  2. Creating RDDs
  3. RDD Operations
  4. Passing Functions to Spark
  5. Common Transformations and Actions
  6. Persistence (Caching)
  7. Conclusion
  1. Motivation
  2. Creating Pair RDDs
  3. Transformations on Pair RDDs
  4. Actions Available on Pair RDDs
  5. Data Partitioning (Advanced)
  6. Conclusion
  1. Motivation
  2. File Formats
  3. Filesystems
  4. Structured Data with Spark SQL
  5. Databases
  6. Conclusion
  1. Introduction
  2. Accumulators
  3. Broadcast Variables
  4. Working on a Per-Partition Basis
  5. Piping to External Programs
  6. Numeric RDD Operations
  7. Conclusion
  1. Introduction
  2. Spark Runtime Architecture
  3. Deploying Applications with spark-submit
  4. Packaging Your Code and Dependencies
  5. Scheduling Within and Between Spark Applications
  6. Cluster Managers
  7. Which Cluster Manager to Use?
  8. Conclusion
  1. Configuring Spark with SparkConf
  2. Components of Execution: Jobs, Tasks, and Stages
  3. Finding Information
  4. Key Performance Considerations
  5. Scheduling Within and Between Spark Applications
  6. Conclusion
  1. Linking with Spark SQL
  2. Using Spark SQL in Applications
  3. Loading and Saving Data
  4. JDBC/ODBC Server
  5. User-Defined Functions
  6. Spark SQL Performance
  7. Conclusion
  1. A Simple Example
  2. Architecture and Abstraction
  3. Transformations
  4. Output Operations
  5. Input Sources
  6. 24/7 Operation
  7. Streaming UI
  8. Performance Considerations
  9. Kafka Basics
  10. Conclusion
  1. Overview
  2. System Requirements
  3. Machine Learning Basics
  4. Data Types
  5. Algorithms
  6. Tips and Performance Considerations
  7. Pipeline API
  8. Conclusion

What Certificate do we provide?

Based on your performance in Quizzes, Assignments and Projects, we provide the certificate in the following forms:

1. Hard Copy

We send a hard copy of the certificate to your address.

Digitally Signed 2. Digitally Signed Copy

We provide the PDF of the certificate that is digitally signed by KnowBigData.com.

3. Share Your Success

Share your course record with employers and educational institutions through a secure permanent URL.

LinkedIn Recommendation & Endorsements 4. LinkedIn Recommendation & Endorsements

We will provide a LinkedIn Recommendation based on your performance. Also, we will endorse you with tags such as Hadoop, Big Data.

Verifiable Certificate 5. Verifiable Certificate

We have provided an online form to validate whether the certificate is correct or not here. This assists recruiters to verify the certificate provided by us.

About the Team

Sandeep Giri

Sandeep Giri

Founder & Chief Instructor

Past Amazon.com, InMobi.com, Founder @ tBits Global, D.E.Shaw

Education Indian Institute of Technology, Roorkee

For last 12 years, Sandeep has been building products and churning large amounts of data for various product firms. He has an all around experience of software development and big data analysis.

Apart from digging data and technologies, Sandeep enjoys conducting interviews and explaining difficult concepts in simple ways.

Read More

Big Data with Apache Spark - Frequently Asked Questions

Yes. Java is not required for understanding this course. We will be covering Spark's Python for programming in Spark. So, if you qualify the following three criteria, :

  1. Basics Of SQL. You should know the basics of SQL and databases. If you know about filters in SQL, you are expected to understand the course.
  2. A know-how of the basics of programming. If you understand 'loops' in any programming language, and if you are able to create a directory and see whats inside a file from the command line, you are good to get the concepts of this course even if you have not really touched programming for the last 10 years! In addition, we will be providing video classes on the basics of Python.

No. We stopped classroom trainings a while back when we realized that our students attending the online instructor led classes are performing better in the assignments than students in our offline classrooms. Moreover, students ask more questions in online sessions in comparison to the classroom sessions.

Also, it is very difficult to get a real training locally in any city. So, it is better to have really good training than having the classroom sessions.

To check if the online session would work for you, please attend our demo sessions. I can assure you that you would like the instructor led online trainings.

There are two ways to do practicals.
  1. Using the our CloudxLabTo give our candidates a real experience of big data computing, we have provided a bunch of computers with all the big data technologies running on them since most of the big data technologies make sense only if done using multiple machines. You only have to use SSH Client (putty on windows) to connect to our cluster. Whether you are at home or office, and whether you are using a laptop or a tablet, you would be able to use Spark. See more details about CloudxLab, here.
  2. Using Virtual MachinesSecond and the traditional way to experiment on Spark is to install a Virtual Machine. We will assist you in setting up Virtual Machine. However, most of our students are so happy with our CloudxLab that they hardly install a Virtual Machine.

Our classes are held every weekend on Saturdays and Sundays either in the mornings or in the evenings. So, there are two classes for 3 hours each; one on saturday and one on sunday.

In addition to the 6 hours of weekend classes, you will have to devote around 4-6 hours every week to complete assignments.

If you are not able to attend a particular class, you can watch the recordings of that class. Otherwise, you can attend the same class in another running batch.

Sometimes, due to various reasons, people find it difficult to continue a course. In case that happens, you can continue in another session in the future, or you can request your refund. Here are the guidelines for requesting the refund.

Yes, the course material is available to our students for life. You will have access to the content in LMS for ever.

Yes, we provide our own Certification. At the end of your course, you will work on a real time project. You will receive a Problem Statement along with a data-set to work on our CloudxLab. Once you are successfully through the project (Reviewed by an expert), you will be awarded a certificate with a performance-based grading.If your project is not approved in the first attempt, you can take extra assistance to understand concepts better and reattempt the project free of cost.

Big Data with Apache Spark is one of the hottest career options available today for software engineers. There are around thousands of jobs currently in U.S. alone for Data Analysts and the demand for Data Analysts is far more than the availability. Learn more about career prospects in Data Analysis at naukri.com and indeed.com.

Our cluster has all the softwares that are required for the course plus some more components such as GIT and R. In case you require a particular software to be installed on cluster which is not already there, please let us know.

Self Paced Classes
Starts Time Details Trainer Price
Self Paced Anytime
  • Self Paced Classes
  • Total 11 Sessions
Sandeep Giri

(20% off - Early Bird) (Incl. Taxes)
Upcoming Live Courses

  Big Data with Hadoop and Spark - Starts on 14th January, 2017