Chapter 2 Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 2, Input: concepts, instances, attributes of Data Mining by I. H. Witten, E. Frank, M. A. Hall and C. J. Pal 2...

1 answer below »
I need you to help me with this assignment


Chapter 2 Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 2, Input: concepts, instances, attributes of Data Mining by I. H. Witten, E. Frank, M. A. Hall and C. J. Pal 2 Input: concepts, instances, attributes • Components of the input for learning • What’s a concept? • Classification, association, clustering, numeric prediction • What’s in an example? • Relations, flat files, recursion • What’s in an attribute? • Nominal, ordinal, interval, ratio • Preparing the input • ARFF, sparse data, attributes, missing and inaccurate values, unbalanced data, getting to know your data 3 Components of the input • Concepts: kinds of things that can be learned • Aim: intelligible and operational concept description • Instances: the individual, independent examples of a concept to be learned • More complicated forms of input with dependencies between examples are possible • Attributes: measuring aspects of an instance • We will focus on nominal and numeric ones 4 What’s a concept? • Concept: thing to be learned • Concept description: output of learning scheme • Styles of learning: • Classification learning: predicting a discrete class • Association learning: detecting associations between features • Clustering: grouping similar instances into clusters • Numeric prediction: predicting a numeric quantity 5 Classification learning • Example problems: weather data, contact lenses, irises, labor negotiations • Classification learning is supervised • Scheme is provided with actual outcome • Outcome is called the class of the example • Measure success on fresh data for which class labels are known (test data) • In practice success is often measured subjectively 6 Association learning • Can be applied if no class is specified and any kind of structure is considered “interesting” • Difference to classification learning: • Can predict any attribute’s value, not just the class, and more than one attribute’s value at a time • Hence: far more association rules than classification rules • Thus: constraints are necessary, such as minimum coverage and minimum accuracy 7 Clustering • Finding groups of items that are similar • Clustering is unsupervised • The class of an example is not known • Success often measured subjectively … … … Iris virginica1.95.12.75.8102 101 52 51 2 1 Iris virginica2.56.03.36.3 Iris versicolor1.54.53.26.4 Iris versicolor1.44.73.27.0 Iris setosa0.21.43.04.9 Iris setosa0.21.43.55.1 TypePetal widthPetal lengthSepal widthSepal length 8 Numeric prediction • Variant of classification learning where “class” is numeric (also called “regression”) • Learning is supervised • Scheme is being provided with target value • Measure success on test data …………… 40FalseNormalMildRainy 55FalseHighHot Overcast 0TrueHighHotSunny 5FalseHighHotSunny Play-timeWindyHumidityTemperatureOutlook 9 What’s in an example? • Instance: specific type of example • Thing to be classified, associated, or clustered • Individual, independent example of target concept • Characterized by a predetermined set of attributes • Input to learning scheme: set of instances/dataset • Represented as a single relation/flat file • Rather restricted form of input • No relationships between objects • Most common form in practical data mining 10 A family tree = Steven M Graham M Pam F Grace F Ray M = Ian M Pippa F Brian M = Anna F Nikki F Peggy F Peter M 11 Family tree represented as a table IanPamFemaleNikki IanPamFemaleAnna RayGraceMaleBrian RayGraceFemalePippa RayGraceMaleIan PeggyPeterFemalePam PeggyPeterMaleGraham PeggyPeterMaleSteven ??FemalePeggy ??MalePeter parent2Parent1GenderName 12 The “sister-of” relation yesAnnaNikki ……… YesNikkiAnna ……… YesPippaIan ……… YesPamSteven NoGrahamSteven NoPeterSteven ……… NoStevenPeter NoPeggyPeter Sister of?Second person First person NoAll the rest YesAnnaNikki YesNikkiAnna YesPippaBrian YesPippaIan YesPamGraham YesPamSteven Sister of?Second person First person Closed-world assumption 13 A full representation in one table Ian Ian Ray Ray Peggy Peggy Parent2 Female Female Female Female Female Female Gender Pam Pam Grace Grace Peter Peter Parent1NameParent2Parent1GenderName Ian Ian Ray Ray Peggy Peggy Pam Pam Grace Grace Peter Peter Female Female Male Male Male Male NoAll the rest YesAnnaNikki YesNikkiAnna YesPippaBrian YesPippaIan YesPamGraham YesPamSteven Sister of? Second personFirst person If second person’s gender = female and first person’s parent = second person’s parent then sister-of = yes 14 Generating a flat file • Process of flattening called “denormalization” • Several relations are joined together to make one • Possible with any finite set of finite relations • Problematic: relationships without a pre-specified number of objects • Example: concept of nuclear-family • Note that denormalization may produce spurious regularities that reflect the structure of the database • Example: “supplier” predicts “supplier address” 15 The “ancestor-of” relation YesOther positive examples here YesIanPamFemaleNikki??FemaleGrace Ray Ian Ian Ian Peggy Peggy Parent2 Male Female Female Female Female Male Gender Grace Pam Pam Pam Peter Peter Parent1NameParent2Parent1GenderName ? Peggy ? ? ? ? ? Peter ? ? ? ? Female Female Male Male Male Male NoAll the rest YesIanGrace YesNikkiPam YesNikkiPeter YesAnnaPeter YesPamPeter YesStevenPeter Ancestor of? Second personFirst person 16 Recursion • Appropriate techniques are known as “inductive logic programming” (ILP) methods • Example ILP method: Quinlan’s FOIL rule learner • Problems: (a) noise and (b) computational complexity If person1 is a parent of person2 then person1 is an ancestor of person2 If person1 is a parent of person2 and person2 is an ancestor of person3 then person1 is an ancestor of person3 • Infinite relations require recursion 17 Multi-instance concepts • Each individual example comprises a bag (aka multi-set) of instances • All instances are described by the same attributes • One or more instances within an example may be responsible for the example's classification • Goal of learning is still to produce a concept description • Important real world applications • Prominent examples are drug activity prediction and image classification • A drug can be viewed as bag of different geometric arrangements of the drug molecule • An image can be represented as a bag of image components 18 What’s in an attribute? • Each instance is described by a fixed predefined set of features, its “attributes” • But: number of attributes may vary in practice • Possible solution: “irrelevant value” flag • Related problem: existence of an attribute may depend of value of another one • Possible attribute types (“levels of measurement”): • Nominal, ordinal, interval and ratio 19 Nominal levels of measurement • Values are distinct symbols • Values themselves serve only as labels or names • Nominal comes from the Latin word for name • Example: attribute “outlook” from weather data • Values: “sunny”,”overcast”, and “rainy” • No relation is implied among nominal values (no ordering or distance measure) • Only equality tests can be performed 20 Ordinal levels of measurement • Impose order on values • But: no distance between values defined • Example: attribute “temperature” in weather data • Values: “hot” > “mild” > “cool” • Note: addition and subtraction don’t make sense • Example rule: temperature < hot  play = yes • distinction between nominal and ordinal not always clear (e.g., attribute “outlook”) 21 interval quantities • interval quantities are not only ordered but measured in fixed and equal units • example 1: attribute “temperature” expressed in degrees fahrenheit • example 2: attribute “year” • difference of two values makes sense • sum or product doesn’t make sense • zero point is not defined! 22 ratio quantities • ratio quantities are ones for which the measurement scheme defines a zero point • example: attribute “distance” • distance between an object and itself is zero • ratio quantities are treated as real numbers • all mathematical operations are allowed • but: is there an “inherently” defined zero point? • answer depends on scientific knowledge (e.g., fahrenheit knew no lower limit to temperature) 23 attribute types used in practice • many data mining schemes accommodate just two levels of measurement: nominal and ordinal • others deal exclusively with ratio quantities • nominal attributes are also called “categorical”, ”enumerated”, or “discrete” • but: “enumerated” and “discrete” imply order • special case: dichotomy (“boolean” attribute) • ordinal attributes are sometimes coded as “numeric” or “continuous” • but: “continuous” implies mathematical continuity 24 metadata • information about the data that encodes background knowledge • in theory this information can be used to restrict the search space of the learning algorithm • examples: • dimensional considerations (i.e., expressions must be dimensionally correct) • circular orderings (e.g., degrees in compass) • partial orderings (e.g., generalization/specialization relations) 25 preparing the input • denormalization is not the only issue when data is prepared for learning • problem: different data sources (e.g., sales department, customer billing department, …) • differences: styles of record keeping, coding conventions, time periods, data aggregation, primary keys, types of errors • data must be assembled, integrated, cleaned up • “data warehouse”: consistent point of access • external data may be required (“overlay data”) • critical: type and level of data aggregation 26 the arff data format % % arff file for weather data with some numeric features % @relation weather @attribute outlook {sunny, overcast, rainy} @attribute temperature numeric @attribute humidity numeric @attribute windy {true, false} @attribute play? {yes, no} @data sunny, 85, 85, false, no sunny, 80, 90, true, no overcast, 83, 86, false, yes ... 27 additional attribute types • arff data format also supports string attributes: • similar to nominal attributes but list of values is not pre-specified • additionally, it supports date attributes: • uses the iso-8601 combined date and time format yyyy-mm-dd- thh:mm:ss @attribute description string @attribute today date 28 relational attributes • relational attributes allow multi-instance problems to be represented in arff format • each value of a relational attribute is a separate bag of instances, but each bag has the same attributes • nested attribute block gives the structure of the referenced instances @attribute bag relational @attribute outlook { sunny, overcast, rainy } @attribute temperature numeric @attribute humidity numeric @attribute windy { true, false } @end bag 29 multi-instance arff % % multiple instance arff file for the weather data % @relation weather @attribute bag_id { 1, 2, 3, 4, 5, 6, 7 } @attribute bag relational @attribute outlook {sunny, overcast, rainy} @attribute temperature numeric @attribute humidity numeric @attribute windy {true, false} @attribute play? {yes, no} @end bag @data 1, “sunny, 85, 85, false\nsunny, 80, 90, true”, no 2, “overcast, 83, 86, false\nrainy, 70, 96, false”, yes ... 30 sparse data • in some applications most attribute values are zero and storage requirements can be reduced • e.g.: word counts in a text categorization problem • arff supports sparse data storage • this also works for nominal attributes (where the first value of the attribute corresponds to “zero”) • some learning algorithms work very efficiently with sparse data 0, 26, 0, 0, 0 ,0, 63, 0, 0, 0, “class a” 0, 0, 0, 42, 0, 0, 0, 0, 0, 0, “class b” {1 26, 6 63, 10 “class a”} {3 42, 10 “class b”} 31 attribute types • interpretation of attribute hot="" ="" play="yes" •="" distinction="" between="" nominal="" and="" ordinal="" not="" always="" clear="" (e.g.,="" attribute="" “outlook”)="" 21="" interval="" quantities="" •="" interval="" quantities="" are="" not="" only="" ordered="" but="" measured="" in="" fixed="" and="" equal="" units="" •="" example="" 1:="" attribute="" “temperature”="" expressed="" in="" degrees="" fahrenheit="" •="" example="" 2:="" attribute="" “year”="" •="" difference="" of="" two="" values="" makes="" sense="" •="" sum="" or="" product="" doesn’t="" make="" sense="" •="" zero="" point="" is="" not="" defined!="" 22="" ratio="" quantities="" •="" ratio="" quantities="" are="" ones="" for="" which="" the="" measurement="" scheme="" defines="" a="" zero="" point="" •="" example:="" attribute="" “distance”="" •="" distance="" between="" an="" object="" and="" itself="" is="" zero="" •="" ratio="" quantities="" are="" treated="" as="" real="" numbers="" •="" all="" mathematical="" operations="" are="" allowed="" •="" but:="" is="" there="" an="" “inherently”="" defined="" zero="" point?="" •="" answer="" depends="" on="" scientific="" knowledge="" (e.g.,="" fahrenheit="" knew="" no="" lower="" limit="" to="" temperature)="" 23="" attribute="" types="" used="" in="" practice="" •="" many="" data="" mining="" schemes="" accommodate="" just="" two="" levels="" of="" measurement:="" nominal="" and="" ordinal="" •="" others="" deal="" exclusively="" with="" ratio="" quantities="" •="" nominal="" attributes="" are="" also="" called="" “categorical”,="" ”enumerated”,="" or="" “discrete”="" •="" but:="" “enumerated”="" and="" “discrete”="" imply="" order="" •="" special="" case:="" dichotomy="" (“boolean”="" attribute)="" •="" ordinal="" attributes="" are="" sometimes="" coded="" as="" “numeric”="" or="" “continuous”="" •="" but:="" “continuous”="" implies="" mathematical="" continuity="" 24="" metadata="" •="" information="" about="" the="" data="" that="" encodes="" background="" knowledge="" •="" in="" theory="" this="" information="" can="" be="" used="" to="" restrict="" the="" search="" space="" of="" the="" learning="" algorithm="" •="" examples:="" •="" dimensional="" considerations="" (i.e.,="" expressions="" must="" be="" dimensionally="" correct)="" •="" circular="" orderings="" (e.g.,="" degrees="" in="" compass)="" •="" partial="" orderings="" (e.g.,="" generalization/specialization="" relations)="" 25="" preparing="" the="" input="" •="" denormalization="" is="" not="" the="" only="" issue="" when="" data="" is="" prepared="" for="" learning="" •="" problem:="" different="" data="" sources="" (e.g.,="" sales="" department,="" customer="" billing="" department,="" …)="" •="" differences:="" styles="" of="" record="" keeping,="" coding="" conventions,="" time="" periods,="" data="" aggregation,="" primary="" keys,="" types="" of="" errors="" •="" data="" must="" be="" assembled,="" integrated,="" cleaned="" up="" •="" “data="" warehouse”:="" consistent="" point="" of="" access="" •="" external="" data="" may="" be="" required="" (“overlay="" data”)="" •="" critical:="" type="" and="" level="" of="" data="" aggregation="" 26="" the="" arff="" data="" format="" %="" %="" arff="" file="" for="" weather="" data="" with="" some="" numeric="" features="" %="" @relation="" weather="" @attribute="" outlook="" {sunny,="" overcast,="" rainy}="" @attribute="" temperature="" numeric="" @attribute="" humidity="" numeric="" @attribute="" windy="" {true,="" false}="" @attribute="" play?="" {yes,="" no}="" @data="" sunny,="" 85,="" 85,="" false,="" no="" sunny,="" 80,="" 90,="" true,="" no="" overcast,="" 83,="" 86,="" false,="" yes="" ...="" 27="" additional="" attribute="" types="" •="" arff="" data="" format="" also="" supports="" string="" attributes:="" •="" similar="" to="" nominal="" attributes="" but="" list="" of="" values="" is="" not="" pre-specified="" •="" additionally,="" it="" supports="" date="" attributes:="" •="" uses="" the="" iso-8601="" combined="" date="" and="" time="" format="" yyyy-mm-dd-="" thh:mm:ss="" @attribute="" description="" string="" @attribute="" today="" date="" 28="" relational="" attributes="" •="" relational="" attributes="" allow="" multi-instance="" problems="" to="" be="" represented="" in="" arff="" format="" •="" each="" value="" of="" a="" relational="" attribute="" is="" a="" separate="" bag="" of="" instances,="" but="" each="" bag="" has="" the="" same="" attributes="" •="" nested="" attribute="" block="" gives="" the="" structure="" of="" the="" referenced="" instances="" @attribute="" bag="" relational="" @attribute="" outlook="" {="" sunny,="" overcast,="" rainy="" }="" @attribute="" temperature="" numeric="" @attribute="" humidity="" numeric="" @attribute="" windy="" {="" true,="" false="" }="" @end="" bag="" 29="" multi-instance="" arff="" %="" %="" multiple="" instance="" arff="" file="" for="" the="" weather="" data="" %="" @relation="" weather="" @attribute="" bag_id="" {="" 1,="" 2,="" 3,="" 4,="" 5,="" 6,="" 7="" }="" @attribute="" bag="" relational="" @attribute="" outlook="" {sunny,="" overcast,="" rainy}="" @attribute="" temperature="" numeric="" @attribute="" humidity="" numeric="" @attribute="" windy="" {true,="" false}="" @attribute="" play?="" {yes,="" no}="" @end="" bag="" @data="" 1,="" “sunny,="" 85,="" 85,="" false\nsunny,="" 80,="" 90,="" true”,="" no="" 2,="" “overcast,="" 83,="" 86,="" false\nrainy,="" 70,="" 96,="" false”,="" yes="" ...="" 30="" sparse="" data="" •="" in="" some="" applications="" most="" attribute="" values="" are="" zero="" and="" storage="" requirements="" can="" be="" reduced="" •="" e.g.:="" word="" counts="" in="" a="" text="" categorization="" problem="" •="" arff="" supports="" sparse="" data="" storage="" •="" this="" also="" works="" for="" nominal="" attributes="" (where="" the="" first="" value="" of="" the="" attribute="" corresponds="" to="" “zero”)="" •="" some="" learning="" algorithms="" work="" very="" efficiently="" with="" sparse="" data="" 0,="" 26,="" 0,="" 0,="" 0="" ,0,="" 63,="" 0,="" 0,="" 0,="" “class="" a”="" 0,="" 0,="" 0,="" 42,="" 0,="" 0,="" 0,="" 0,="" 0,="" 0,="" “class="" b”="" {1="" 26,="" 6="" 63,="" 10="" “class="" a”}="" {3="" 42,="" 10="" “class="" b”}="" 31="" attribute="" types="" •="" interpretation="" of="">
Answered 4 days AfterFeb 11, 2022

Answer To: Chapter 2 Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 2, Input:...

Mohd answered on Feb 16 2022
103 Votes
SOLUTION.PDF

Answer To This Question Is Available To Download

Related Questions & Answers

More Questions »

Submit New Assignment

Copy and Paste Your Assignment Here