BIG Data Management-1


BIG Data Management-1 

Assignment 1 

Due Date: 

Assignment type: Individual 

Submission documents: 

(A word/PDF document with results/screenshots of results, along with the code files to be uploaded on LMS) 

Assignment 1 contains three questions and will ask you to get familiar with aspects of Apache Spark. While first two questions require you to get familiar with Spark programming, the last question will ask you to understand an existing code and explain it in simple terms.

Q1. Consider the two data files (users.csv, transactions.csv). Users file has the following fields: 

a) UserID 

b) EmailID 

c) NativeLanguage 

d) Location 

 Transactions file has the following fields: 

a) Transaction_ID 

b) Product_ID 

c) UserID 

d) Price 

e) Product_Description 

By making use of Spark Core (i.e. without using Spark SQL) find out: 

a) Count of unique locations where each product is sold. 

b) Find out products bought by each user. 

c) Total spending done by each user on each product. 

Remember, you have to make use of Spark Core for this question. (15 Marks)

Q2. Consider the dataset file Olympics.csv. This file contains information about the Olympic games, players participating in the games, and details of medals won by them. Using Spark core and the data file, compute the following:

  1. Total medals that each country won in a particular sport (such as Gymnastics).
  2. In each Olympic games, how many medals has India won?
  3. Compute top 3 countries in terms of total medals by each Olympic games year.

Remember, you have to make use of Spark Core for this question. (10 Marks).

Q3. Consider the Movie Recommendation code and problem that was discussed during the class (Session 5 and 6). Please provide a brief write-up on the problem, steps needed to arrive at the solution (recommendation system), and how exactly those steps are implemented in the code. You can make use of the PPT file that discusses the broad solution. While you are doing so, please also mention what each line of code does (It is not sufficient to mention what each block of code does, you would have to provide explanation for each line). (25 Marks)

The post BIG Data Management-1 appeared first on My Assignment Online.



This essay is written by:

Prof. SirMojo Verified writer

Finished papers: 435

Proficient in:

English, History, Business and Entrepreneurship, Nursing, Psychology, Management

You can get writing help to write an essay on these topics
100% plagiarism-free

Hire This Writer
© 2017 theacademicessays. All Rights Reserved. Design & Developed by theacademicessays.
How to Avoid Plagiarism
  • Use multiple resourses when assembling your essay
  • Use Plagiarism Checker to double check your essay
  • Get help from professional writers when not sure you can do it yourself
  • Do not copy and paste free to download essays
Get plagiarism free essay
Loading...