The assessment is about AWS cloud system development
your task is to implement a Lambda function that is invoked after a “Student.json” object is created in an S3 bucket. Upon execution, the Lambda function read the content in the “Student.json” file and inserts them into a DynamoDB table. In Part B, your task also includes preparing a dataset by applying pre-processing techniques. Your submission will be evaluated based on:
PROG2003 – Cloud Systems Development Assignment 2 PROG2003 Assignment 2 Weight: 40% of your final mark Due: Week 6 Monday (5 December 2022) at 11:00 PM Specifications In Part A, your task is to implement a Lambda function that is invoked after a “Student.json” object is created in an S3 bucket. Upon execution, the Lambda function reads the content in the “Student.json” file and inserts them into a DynamoDB table. In Part B, your task also includes preparing a dataset by applying pre-processing techniques. Your submission will be evaluated based on: • whether instructions have been followed; • correct functionality; • correct implementation; and • comments inside the program Getting Help This assignment, which is to be completed individually, is your chance to gain an understanding of the fundamental concepts of Lambda and Lambda events on which later learning will be based. It is important that you master these concepts yourself. Since you are mastering fundamental skills, you are permitted to work from the examples in the MySCU site or textbook, but you must acknowledge assistance from other textbooks or classmates. In particular, you must not use online material or help from others, as this would prevent you from mastering these concepts. This diagram will help you understand where you can get help: Encouraged Attribution Required Not acceptable Ask tutor Be aware that if you do get help from one of the red sources, you will be reported for academic misconduct, which may have serious penalties. Please visit the following link for the guidelines: https://bit.ly/scuAcadMisconduct Lecturer Tutors Online Forums Relatives Students outside unit Hired coders Classmates Private Tutors Other https://bit.ly/scuAcadMisconduct PROG2003 – Cloud Systems Development Assignment 2 ATTENTION: Assignment 2 must be completed in the UA-provided AWS account. Personal AWS accounts will not be accepted, and there will be a significant mark deduction. If your AWS account is not accessible for any reason and the marker cannot check your app, you will also lose significant marks. No excuse will be considered. You must use Java and relevant AWS SDK to implement your app. Details: The details of the assignment tasks are given below. Part A: (30 marks): • Create an S3 bucket named “yourscuusername-a2-bucket”. • Implement a Lambda function named “yourscuusername-a2-function”. Add a “create object” trigger for the bucket “yourscuusername-a2-bucket” to this Lambda function. • Use the “LabRole” as the execution role of the Lambda function. Add an inline policy to this role so that “LabRole” allows DynamoDB actions for all resources. If you find that an inline policy is already added to your “LabRole” for DynamoDB access and it works, then you can use that; no need to create a new one. • The Lambda function is invoked only when a new object named “Student.json” is created/uploaded in “yourscuusername-a2-bucket”. The “Student.json” includes a JSON array with more than one JSON objects. An example of the JSON array is given below. A sample Student.json is attached with this assignment. You can add new elements to this JSON file. [{"Name": "Jacob", "Age": 21, "Degree": "Bachelor of IT", "Credits": 24}, {"Name": "Sarah", "Age": 18, "Degree": "Bachelor of IT", "Credits": 30}, {"Name": "Marcus", "Age": 24, "Degree": "Bachelor of IT", "Credits": 36}, {"Name": "Peter", "Age": 29, "Degree": "Master of IT", "Credits": 40}, {"Name": "Tori", "Age": 19, "Degree": "Master of IT", "Credits": 12}] • Create a Cloud9 app named “yourscuusernameA2App”. Implement a handler named “handleRequest()” with appropriate parameters to receive S3 event notifications. • The handler retrieves the bucket name and object key associated with the S3 event and conducts the following tasks only for the “Student.json” object. For any other object created/uploaded the handler will not do anything following. • The handler reads the JSON array in the “Student.json”, parses the JSON objects, and inserts the values of the fields to a DynamoDB table named “Student”. The schema of the table is given below, where Id is the primary key. You can add a test item to the table as shown below. Student(Id: Number, Name: String, Age: Number, Degree: String, Credits: Number) • The handler must retrieve the latest value of “Id” from the DynamoDB table before doing any insertion. • You must use AWS SDK v1.x, appropriate exception handlings, loops, and separate methods for insertion and reading the latest Id. • Add detailed comments to your Java source code. JSON parsing must be done by “org.json” package. PROG2003 – Cloud Systems Development Assignment 2 Testing the Lambda function: You can test the Lambda function by just simply uploading the Student.json into the specified bucket. Solution Helps: Check the following hints for implementing Part A. • You need to use the “org.json” package to retrieve the JSON array and parse each JSON object. Please check this link: https://stleary.github.io/JSON-java/index.html. You can use the latest maven dependency from here: https://mvnrepository.com/artifact/org.json/json. • A JSONArray is a collection of JSONObjects and a JSONObject is a collection of values of each data filed. You need to understand this concept and parse the fields. Part B: (10 marks): • You need to create, publish, and import a recipe named “yourscuusername-recipe” to complete the following tasks on the “resolution.csv” (attached with this assignment) in the Glue DataBrew project. Create a project named “yourscuusernameA2Project”. • Replace all missing values in any two columns which have missing values. Use the most frequent value and numeric aggregate to compute the missing value. • Identify the duplicate rows and flag them with “yes or no” values. • Flag outliers for any one column as “yes or no” values. • Compute the total number of “no votes” for each distinct “vote date” and add a new column to the dataset. • Compute the total number of “yes votes” for each distinct “vote date” and add a new column to the dataset. • Use “LabRole” while creating the project. Upload the “resolution.csv” to “yourscuusername- a2-bucket”. • Import the recipe as a JSON file. Solution Helps: No solution hint is required for implementing Part B. Submission Checklist: The marker will access your AWS Academy workspace (provided by the UA), check your app, and directly mark your app from there. You have to zip the app project folder and submit it in Blackboard as well. Blackboard submission list: • Zipped app folder for the Handler app. • The recipe JSON file. https://stleary.github.io/JSON-java/index.html https://mvnrepository.com/artifact/org.json/json