Note you must use RapidMiner Studio for Task 1 in this Assignment 3. Assignment 3 Report for Tasks 1 and 2 in Word document format with extension .docx
ADA Compliant Lecture PowerPoint Module 9- Big Data Concepts and Technologies Part 2 CIS8008 Business Intelligence Slide # of total Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Module 9 Learning Objectives On successful completion of this module: compare and contrast complementary uses of data warehousing and big data describe the main big data platforms and services describe and understand the need, importance and application of the stream analytics Slide # of total Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Module 9 Learning Resources Chapter 7 pp. 419–438, Sharda, R, Delen, D & Turban, E,2018, Business Intelligence, Analytics and Data Science: A Managerial Perspective,4th edn, Prentice Hall, Boston, Massachusetts. Slide # of total Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Application Case 7.4 Understanding Quality and Reliability of Healthcare Support Information on Twitter Questions for Discussion 1. What was the data scientists’ main concern regarding health information that is disseminated on the Twitter platform? The main concern was the quality and accuracy of the information that was being offered. 2. How did the data scientists ensure that nonexpert information disseminated on social media could indeed contain valuable health information? The scientists evaluated the type of information that was being presented, and specifically how it was supported. Information that was supported using more objective data tended to be more accurate than information supported only by opinion. 3. Does it make sense that influential users would share more objective information whereas less influential users could focus more on subjective information? Why? It is possible that influential users are more influential because they are more accurate. By sticking to objective data, influential users are able to provide higher quality information, and that is the type of information that users will be looking for. Slide 7-4 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Big Data and Data Warehousing What is the impact of Big Data on DW? Big Data and RDBMS do not go nicely together Will Hadoop replace data warehousing/RDBMS? Use Cases for Hadoop Hadoop as the repository and refinery Hadoop as the active archive Use Cases for Data Warehousing Data warehouse performance Integrating data that provides business value Interactive BI tools Slide 7-5 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Hadoop versus Data Warehouse When to Use Which Platform Slide 7-6 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Coexistence of Hadoop and DW Use Hadoop for storing and archiving multi-structured data Use Hadoop for filtering, transforming, and/or consolidating multi-structured data Use Hadoop to analyze large volumes of multi-structured data and publish the analytical results Use a relational DBMS that provides MapReduce capabilities as an investigative computing platform Use a front-end query tool to access and analyze data Slide 7-7 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Coexistence of Hadoop and DW Source: Teradata Slide 7-8 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Big Data Vendors Big Data vendor landscape is developing very rapidly A representative list would include Cloudera - cloudera.com MapR – mapr.com Hortonworks - hortonworks.com Also, IBM (Netezza, InfoSphere), Oracle (Exadata, Exalogic), Microsoft, Amazon, Google, … Software, Hardware, Service, … Slide 7-9 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved IBM InfoSphere BigInsights Slide 7-10 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Application Case 7.5 –Predicting Flu outbreaks Using Social Media for Nowcasting the Flu Activity Questions for Discussion 1. Why would social media be able to serve as an early predictor of flu outbreaks? It is possible to capture information from a wide variety and large number of users in real time. 2. What other variables might help in predicting such outbreaks? Other information such as doctor visits and other health records could be used to validate social media information. 3. Why would this problem be a good problem to solve using Big Data technologies mentioned in this chapter? Because of the large impact on both people and productivity each year due to the flu, better understanding of its spread is very helpful. Big Data technologies are an excellent way to access and understand this type of raw data from social media systems. Slide 7-11 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Big Data Platforms Teradata Aster Slide 7-12 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Application Case 7.6 (1) Analyzing Disease Patterns from an Electronic Medical Records Data Warehouse Questions for Discussion 1. Why could comorbidity of diseases be different between rural and urban hospitals? Based on the case, there are two options. One option is that people in urban areas may have more disease conditions because they are closer in proximity with other individuals. A second option is that individuals in rural areas may not be diagnosed as frequently with other diseases because of the conditions in those hospitals. 2. What is the issue about the huge difference between rural and urban patient encounters? This may be explained because row locations are less likely to be able to afford complex IT solutions that could capture this data for analysis. Slide 7-13 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Application Case 7.6 (2) Analyzing Disease Patterns from an Electronic Medical Records Data Warehouse Questions for Discussion 3. What are the main components of a network? Important components of the network model include nodes and edges. Nodes identify unique descriptions, in this case diseases, and edges connect similar nodes. 4. Where else can you apply the network approach? Student responses will vary, but may include a mapping of customer preferences, and how groups of preferences may indicate particular markets for goods or services. Slide 7-14 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved FIGURE 7.11 Urban and Rural Comorbidity Networks Slide 7-15 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Technology Insights 7.3 How to Succeed with Big Data Simplify Coexist Visualize Empower Integrate Govern Evangelize Slide 7-16 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Big Data And Stream Analytics Data-in-motion analytics and real-time data analytics One of the Vs in Big Data = Velocity Analytic process of extracting actionable information from continuously flowing data Why Stream Analytics? It may not be feasible to store the data, or lose its value Stream Analytics Versus Perpetual Analytics Critical Event Processing? Slide 7-17 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Stream Analytics A Use Case in Energy Industry Slide 7-18 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Stream Analytics Applications e-Commerce Telecommunication Law Enforcement and Cyber Security Power Industry Financial Services Health Services Government Slide 7-19 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Application Case 7.7 Salesforce Is Using Streaming Data to Enhance Customer Value Questions for Discussion 1. Are there areas in any industry where streaming data is irrelevant? According to the case, understanding of streaming data has the possibility of being relevant in all industries. 2. Besides customer retention, what are other benefits of using predictive analytics? Predictive analytics can be used in a wide variety of applications. One of these can be tailoring the sales process to meet the needs and decision-making style of potential customers. Slide 7-20 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Sensor Data (Energy Production System Status) Meteorological Data (Wind, Light, Temperature, etc.) Usage Data (Smart Meters, Smart Grid Devises) Permanent Storage Area Streaming Analytics (Predicting Usage, Production and Anomalies) Energy Production System (Traditional and Renewable) Energy Consumption System (Residential and Commercial) Data Integration and Temporary Staging Capacity Decisions Pricing Decisions ADA Compliant Lecture PowerPoint Module 10 - Future Trends in Business Intelligence and Analytics – Part 1 CIS8008 Business Intelligence Slide # of total Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Module 10 Learning Objectives On successful completion of this module: describe emerging technologies that will significantly impact business intelligence and analytics describe the current and future use of cloud computing in business analytics describe how geospatial and location-based analytics supporting organisational decision making Slide # of total Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Module 10 Learning Resources Chapter 8 Future Trends, Privacy and Managerial Considerations in Analytics, pp. 445–475, Sharda, R, Delen, D & Turban, E,2018, Business Intelligence, Analytics and Data Science: A Managerial Perspective,4th edn, Prentice Hall, Boston, Massachusetts. Slide # of total Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved OPENING VIGNETTE/Case Study Analysis of Sensor Data Helps Siemens Avoid Train Failures Slide 8-4 Discussion Questions 1. In industrial equipment such as trains, what parameters might one measure on a regular basis to estimate the equipment’s current performance and future repair needs? There are many parameters that could be evaluated to help estimate current performance and repair needs. Some of these parameters could include time in use, weather, adverse impacts, and so on. 2. How would weather data be useful in analyzing a train’s equipment status? Weather data could indicate if the components have been exposed to water, or if the components have been exposed to excesses and heat or cold. Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved OPENING VIGNETTE Analysis of Sensor Data Helps Siemens Avoid Train Failures Slide 8-5 Discussion Questions 3. Estimate how much data you might collect in one month using, say, 1,000 sensors on a train. Each sensor might yield 1 KB data per second. 1,000 sensors at 1KB of data per second (43,200 K/month) is a total of 43.2 GB across all sensors. 4. How would you propose to store such data sets? This volume of data would need to be stored in a robust database system that would be able to analyze all of the individual readings. Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Internet of Things (IoT) IoT is an area with explosive growth Connecting physical world to the Internet Social Network versus IoT human-to-human vs. machine-to-machine Enablers: sensors and sensing devices Example Self driving cars Fitness trackers Smartbin – trash detectors detecting fill levels Smart refrigerators, and other appliances Slide 8-6 Copyright © 2018, 2014, 2011 Pearson Education, Inc. All Rights Reserved Internet of Things (IoT) By 2020, besides computing and communication devices (tablets, phones, and PCs),