CST 2101 – Business Intelligence Programming Project 1: Text Analysis Background Submission (28 marks) Text analysis and the statistical distribution of words can tell us a lot about who wrote it and...

python pro


CST 2101 – Business Intelligence Programming Project 1: Text Analysis Background Submission (28 marks) Text analysis and the statistical distribution of words can tell us a lot about who wrote it and such an analysis of corporate emails was a key factor in the prosecution of Enron corporation executives for insider trading. Specifically, the increasingly frequent use of pronouns in email was cited as a metric for detecting deceptive communication. The client wants you to expose some concepts about processing text so that they can determine how they to better analyze corporate documents as a first step. Since their internal documents are sensitive, they have asked you to analyze the text copy of the novel: Charles Dickens’ A Tale of Two Cities. You will need to have user interaction and the ability to restart at any time. The user must also be prevented from completing an action that is invalid. For example, the user cannot analyze the text until the file is opened.. 1. Open the text document (must be a valid text document) 2. Preview the document by showing the first 10 lines of text to the user. 3. Choose whether some words will be excluded (for extended capability only) 4. Analyze and display the results 5. Option to save the results to a file The file is sourced from the Gutenberg Project (www.gutenberg.org). You will submit the following files compressed into a 7-zip archive. 1. All python files for your program implementation. 2. Design and Test documents (4) 3. The output text files (1) Submit your file on the LMS under the Project . The zip archive name must be formatted as follows: -2101-Project1TEXT.zip Example: conr0048-2101-Project1TEXT.zip Marking Guide The assignment will be marked out of 28 using the following guide: · The submission follows the project instructions. · The submission provides user interaction for each step with the ability to exit or restart at any point. A graphical user interface may be used. (must provide feedback/notifications to users where appropriate). · The submission demonstrates the correct use of classes/methods · The submission demonstrates the correct use of loops, containers (e.g. lists) and conditional logic. · The program does not contain any logic or runtime errors. · Proper naming conventions for variables, classes etc. · Sufficient header and inline documentation. · Program Design documents are easy to read and reflects your general logic. · Test plan covers basic functionality and exception handling You will be required to perform a demonstration of your project. Project Area Requirement General programming · Must use Python 3, with meaningful naming and consistent style. · Must contain header documentation that describes the purpose of the program, the author and the date. · Must contain sufficient inline documentation for others to understand logic. · Must properly use classes, loops, lists etc. in the program. Design – User interaction · User interaction must be described for each section as well as how the user will be able to return to start or exit. (a flowchart could be used here.) · If using a GUI, must include identification of input and output widgets. · May be designed in Excel, Word, or PDF. · Must be named -Project1-Design-User. Design - Program · Must identify the styles you are adopting · Must covey the general logic of the program including classes/methods. · You may use any combination of pseudocode, narrative, UML diagrams that you wish as long as the logic is clear. · Must be named - Project2-Design-Program. Development and Testing · A small paragraph for development and testing purposes must be used. The file must be named -Project1-TestSample.txt · Must complete a final test plan named -Project1-TestPlan-Final.xls Input · Must accept text files for input and identify invalid file types · Must allow the user to choose the text file for processing. · Must allow the user to choose to start processing or exit without processing any text files Processing · Must be able to process a text file as input: · May allow for some words such as “the”, “and”, to be excluded from the analysis. · Must be able to count the: · total # of words · total # of occurrences for each word. · total # of characters in the text document. · total # of blank spaces in the text document. · Must be able to calculate the percentage of blank spaces as: · total # of blank spaces divided by total # of characters, multiplied by 100. · Must implement exception handling paying close attention to reading and writing operations. Output · Must create an output text file for the chosen text formatted to the client’s requirements using the filename of the input file "-Project1-Analysis.txt". · Must advise the user when: · processing has completed AND · the name/location of the output file. · Must advise the user when: · an exception has occurred AND · the type of exception Client Text Format Requirement · Text analysis output must be clear and easy to read in aligned columns · Must include appropriate headers and data for: · Name of text: · Total Non-blank Character Count: · Total Blank Character Count: · Percentage Blank Character: · Total Word Count: · Word, Count: (each word and count is on a separate line) Book the First--Recalled to Life I. The Period It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair, we had everything before us, we had nothing before us, we were all going direct to Heaven, we were all going direct the other way-- in short, the period was so far like the present period, that some of its noisiest authorities insisted on its being received, for good or for evil, in the superlative degree of comparison only. There were a king with a large jaw and a queen with a plain face, on the throne of England; there were a king with a large jaw and a queen with a fair face, on the throne of France. In both countries it was clearer than crystal to the lords of the State preserves of loaves and fishes, that things in general were settled for ever. It was the year of Our Lord one thousand seven hundred and seventy-five. Spiritual revelations were conceded to England at that favoured period, as at this. Mrs. Southcott had recently attained her five-and-twentieth blessed birthday, of whom a prophetic private in the Life Guards had heralded the sublime appearance by announcing that arrangements were made for the swallowing up of London and Westminster. Even the Cock-lane ghost had been laid only a round dozen of years, after rapping out its messages, as the spirits of this very year last past (supernaturally deficient in originality) rapped out theirs. Mere messages in the earthly order of events had lately come to the English Crown and People, from a congress of British subjects in America: which, strange to relate, have proved more important to the human race than any communications yet received through any of the chickens of the Cock-lane brood. France, less favoured on the whole as to matters spiritual than her sister of the shield and trident, rolled with exceeding smoothness down hill, making paper money and spending it. Under the guidance of her Christian pastors, she entertained herself, besides, with such humane achievements as sentencing a youth to have his hands cut off, his tongue torn out with pincers, and his body burned alive, because he had not kneeled down in the rain to do honour to a dirty procession of monks which passed within his view, at a distance of some fifty or sixty yards. It is likely enough that, rooted in the woods of France and Norway, there were growing trees, when that sufferer was put to death, already marked by the Woodman, Fate, to come down and be sawn into boards, to make a certain movable framework with a sack and a knife in it, terrible in history. It is likely enough that in the rough outhouses of some tillers of the heavy lands adjacent to Paris, there were sheltered from the weather that very day, rude carts, bespattered with rustic mire, snuffed about by pigs, and roosted in by poultry, which the Farmer, Death, had already set apart to be his tumbrils of the Revolution. But that Woodman and that Farmer, though they work unceasingly, work silently, and no one heard them as they went about with muffled tread: the rather, forasmuch as to entertain any suspicion that they were awake, was to be atheistical and traitorous. In England, there was scarcely an amount of order and protection to justify much national boasting. Daring burglaries by armed men, and highway robberies, took place in the capital itself every night; families were publicly cautioned not to go out of town without removing their furniture to upholsterers' warehouses for security; the highwayman in the dark was a City tradesman in the light, and, being recognised and challenged by his fellow-tradesman whom he stopped in his character of "the Captain," gallantly shot him through the head and rode away; the mail was waylaid by seven robbers, and the guard shot three dead, and then got shot dead himself by the other four, "in consequence of the failure of his ammunition:" after which the mail was robbed in peace; that magnificent potentate, the Lord Mayor of London, was made to stand and deliver on Turnham Green, by one highwayman, who despoiled the illustrious creature in sight of all his retinue; prisoners in London gaols fought battles with their turnkeys, and the majesty of the law fired blunderbusses in among them, loaded with rounds of shot and ball; thieves snipped off diamond crosses from the necks of noble lords at Court drawing-rooms; musketeers went into St. Giles's, to search for contraband goods, and the mob fired on the musketeers, and the musketeers fired on the mob, and nobody thought any of these occurrences much out of the common way. In the midst of them, the hangman, ever busy and ever worse than useless, was in constant requisition; now, stringing up long rows of miscellaneous criminals; now, hanging a housebreaker on Saturday who had been taken on Tuesday; now, burning people in the hand at Newgate by the dozen, and now burning pamphlets at the door of Westminster Hall; to-day, taking the life