Lab Work making complex decissions 2021/Report EN.doc Report of laboratory work “Making complex decisions” Riga Technical University Faculty of Computer Science and Information Technology Institute of...

Lab work


Lab Work making complex decissions 2021/Report EN.doc Report of laboratory work “Making complex decisions” Riga Technical University Faculty of Computer Science and Information Technology Institute of Applied Computer Systems Subject “Artificial Intelligence” „LAboratory work: Making complex decisions” Laboratory work report Accomplished by: [Name Surname] Examined by: Prof. J.Grundspenkis Riga, 2013 Absrtact [Your text here] Table of contents 41. Making complex decisions – theoretical background 52. Stepwise algorithm execution 63. Comparison of algorithms in different environments 74. Comparison of algorithms in the same environment 85. Conclusions 9REFEREnces 1. Making complex decisions – theoretical background 1. Shortly describe the essence of making complex decisions. Use illustration if needed. 2. In your own words describe state transition model, utility of state and optimal policy. Do not use definitions found in literature. Use our own example if necessary. 3. Describe value iteration algorithm and write it in pseudo-code. 4. Describe policy iteration algorithm and write it in pseudo-code. 5. Compare value iteration and policy iteration algorithms, describe common and distinctive features. 2. Stepwise algorithm execution Choose an environment with size 3x3 that contains 2 terminal states with sufficiently different rewards (e.g. -1 and 1) and one obstacle. Both terminal states must be reachable. Manually perform two iterations of value iteration algorithm in this environment. Show your calculations as well as their results. Use the provided software to complete these two iterations one-by-one by pressing button „One iteration”. Compare your calculation results with those obtained from the software (they should match). Afterwards use the software to execute algorithm iterations one-by-one until there are no changes in the calculated policy. Repeat the same process (using software only) with the policy iteration algorithm. Assess and compare results of both algorithms. Save the environment configuration into file using the software provided. 3. Comparison of algorithms in different environments Compare results of value iteration and policy iteration algorithm using the software provided. Conduct experiments in environments with sizes 5x5, 10x10, 15x15, 20x20 and 25x25. Insert two terminal states with sufficiently different rewards in each environment. Execute both algorithms in each environment and write down both algorithm execution time and iterations performed. Since the execution time can vary due to some reasons (e.g. programs running in background), each experiment must be repeated at least 5 times. Use the average (arithmetic mean) as the final value. Present your results in tabular as well as graph format. Graphs should represent average execution time and iterations performed for both algorithms depending on the size of the environment. Compare and explain your results. Save each environment in a separate file. 4. Comparison of algorithms in the same environment Choose an environment with size 15x15 with at least two terminal states (with different rewards) and 50-70 obstacles, thus forming a maze where the exit is marked as a terminal state with positive reward and a few other undesirable states (such as dead-ends) are marked as terminal states with negative rewards. Use the software provided to compare the results of value iteration algorithms. Analyze algorithm execution time, iterations performed as well as policies obtained by both algorithms. What and how significant are the differences? How do policy evaluation count and maximum error settings influence the results? Each time you create and change environment or its settings, save it in a separate file. 5. Conclusions [Your text here] REFEREnces [Your text here] © RTU, 2013. 9 Lab Work making complex decissions 2021/software/en/ReinforcementLearning.resources.dll Lab Work making complex decissions 2021/software/lv/ReinforcementLearning.resources.dll Lab Work making complex decissions 2021/software/ReinforcementLearning.exe Lab Work making complex decissions 2021/software/ReinforcementLearning.exe.config en Lab Work making complex decissions 2021/Task EN.doc Description of laboratory work “Making complex decisions”. Riga Technical University Faculty of Computer Science and Information Technology Institute of Applied Computer Systems Subject “ARtificial Intelligence” „LAboratory work: making complex decisions” Laboratory work tasks Riga, 2013 Abstract This document is a description of a laboratory work “Making complex decisions” in the course “Artificial Intelligence”. The document contains description of the laboratory work steps and control tasks. The described laboratory work is intended for 1st course master students who study course “Artificial Intelligence” in the Faculty of Computer Science and Information Technology. Description of the Laboratory work 1. Objective of the work Objective of the work is introducing students to complex decision making in the context of artificial intelligence. 2. Preparing for work To prepare for this laboratory work student should read Chapter 17 (pages 613-625) from the book: · Russel S., Norvig P. Artificial Intelligence, A Modern Approach. Second Edition – New Jersey: Pearson Education, Inc. – 2003. Additionally student should read the software user manual [File UserGuide.doc]. Report form [File Report.doc] must be used as a template for the work. Before proceeding with work, student must apply for this laboratory work at least a week before sending the completed work via emailing to [email protected] with subject “Application to laboratory work: making complex decisions, ”. Fulfilling this requirement guarantees evaluation of your work in “reasonable time” (usually within 2 working days). No guaranties regarding evaluation time are given otherwise. 3. Work steps and control tasks Laboratory work consists of the following five steps (tasks). Student should accomplish all of them to successfully complete this laboratory work. 1. Make a theoretical description about making complex decisions following the given formula: 1. Shortly describe the essence of making complex decisions. Use illustration if needed. 2. In your own words describe state transition model, utility of state and optimal policy. Do not use definitions found in literature. Use our own example if necessary. 3. Describe value iteration algorithm and write it in pseudo-code. 4. Describe policy iteration algorithm and write it in pseudo-code. 5. Compare value iteration and policy iteration algorithms, describe common and distinctive features. 2. Choose an environment with size 3x3 that contains 2 terminal states with sufficiently different rewards (e.g. -1 and 1) and one obstacle. Both terminal states must be reachable. Manually perform two iterations of value iteration algorithm in this environment. Show your calculations as well as their results. Use the provided software to complete these two iterations one-by-one by pressing button „One iteration”. Compare your calculation results with those obtained from the software (they should match). Afterwards use the software to execute algorithm iterations one-by-one until there are no changes in the calculated policy. Repeat the same process (using software only) with the policy iteration algorithm. Assess and compare results of both algorithms. Save the environment configuration into file using the software provided. 3. Compare results of value iteration and policy iteration algorithm using the software provided. Conduct experiments in environments with sizes 5x5, 10x10, 15x15, 20x20 and 25x25. Insert two terminal states with sufficiently different rewards in each environment. Execute both algorithms in each environment and write down both algorithm execution time and iterations performed. Since the execution time can vary due to some reasons (e.g. programs running in background), each experiment must be repeated at least 5 times. Use the average (arithmetic mean) as the final value. Present your results in tabular as well as graph format. Graphs should represent average execution time and iterations performed for both algorithms depending on the size of the environment. Compare and explain your results. Save each environment in a separate file. 4. Choose an environment with size 15x15 with at least two terminal states (with different rewards) and 50-70 obstacles, thus forming a maze where the exit is marked as a terminal state with positive reward and a few other undesirable states (such as dead-ends) are marked as terminal states with negative rewards. Use the software provided to compare the results of value iteration algorithms. Analyze algorithm execution time, iterations performed as well as policies obtained by both algorithms. What and how significant are the differences? How do policy evaluation count and maximum error settings influence the results? Each time you create and change environment or its settings, save it in a separate file. 5. Provide conclusions about your results. Compare results of both algorithms. Which algorithm under what conditions performs faster? Which one is more precise? How does accuracy of utility calculation affect calculated policies? 4. Submission Make sure you use document [File Report.doc] as a template for this work. After the completion of the laboratory work the following files must be submitted in the task “Submission of the fifth laboratory work” at the ORTUS course: · Document containing your report, renamed to match your student ID (e.g. “123456abcd.doc”). · All files created with the provided software while performing steps 2 – 4. Indicate step number and environment size in the file names (e.g. “step 4 15x15.cfg”). 5. Questions and uncertainties, software bugs, etc. Feel free to ask any questions regarding the laboratory work by sending email to [email protected]. Please report any bugs you find in the provided software. Before submitting a bug-report, please consult the user manual [File UserGuide.doc] for details on software operation. A bug-report should include a brief description, a screenshot (if possible), expected and actual behavior of the software, as well as steps to reproduce the problem, and also an indication of importance of the bug in scale 1 – 5, where 1 means minor bug (e.g. misspelling of some word) that does not affect your work and 5 – a groundbreaking bug that prevents you from completing your work. Intermediate values (such as 2.5) are also applicable. Please note that a software bug is not a justifiable reason for you to skip some or all the parts of this work. If you find a groundbreaking bug, please report it ASAP, so it can be fixed and you can continue with your work. 6. Work evaluation Evaluation of the laboratory work is based on the analysis of submitted report as well as verification of the files created with the provided software. Quality of theoretical foundation, conclusions and assessments as
May 24, 2021
SOLUTION.PDF

Get Answer To This Question

Related Questions & Answers

More Questions »

Submit New Assignment

Copy and Paste Your Assignment Here