Data and Information Retrieval Coursework

220 CT Coursework 2020/2021

This is an induvial assignment.  Please type everything neatly in a MS word file and submit to the Canvas System on time.

There are 5 tasks in this CW. They are all mandatory – Online submission no later than 23:59 on the last week of semester. This CW represents 50% of your total module mark.

Question #1: Database Design (This task is worth 20 marks)

Introduction

The International Space Station (ISS) is a habitable artificial satellite in low Earth orbit. It is the ninth space station to be inhabited by crews following previous orbital stations that were launched by the US the former Soviet Union and later Russia. The ISS is intended to be a laboratory, observatory and factory in space as well as to provide transportation, maintenance, and act as a staging base for possible future missions to the Moon, Mars and beyond. In order to support the crew and overall operation of ISS the space agencies in charge of running the station conduct regular missions to launch spacecraft carrying payloads of essential or replacement equipment up to ISS. A payload inventory, see table below, is recorded of each mission, consisting of the space agency leading the mission and the equipment payload to be sent up to ISS. The overall weight of the payload is also determined in order to calculate the fuel needed for orbital insertion of the spacecraft to successfully rendezvous with ISS.

Currently there is no database being used for managing the payload inventory information in the table above. It is therefore necessary to convert the inventory table into a set of database relations by applying and showing the process of normalization to determine the correct relations.

You are required to normalize the current data inventory to third normal form to produce appropriate relations (tables) for the database by providing the steps from UNF, 1NF, 2NF, to 3NF.It is also required to specify the primary and foreign keys for the 3NF tables. You will then be asked to implement the DB using a database of your choice (MySQL recommended).A script of database creation is required to be submitted too with record contents. The data contained inside the database needs to be retrieved by at least one simple query by mission no.Query script and screen dump of script execution would be expected to be submitted as evidence of successful query execution.

Requirements:

Activity 1: Put data in First Normal Form: Remove Repeating Elements or Groups of Elements in Data (3 marks)

Activity 2:  Put data in Second Normal Form:  Remove Partial Dependencies on a Concatenated Key in Data (3 marks)

Activity 3: Put data in Third Normal Form: Remove Dependencies on Non-Key Attributes / Final Database Design (4 marks)

Database creation script with integrity control (i.e. foreign keys): 7 marks

Data insertion / preparation script: 1.5 marks

Query script: 1.5 marks

Question 2– Database and Information Retrieval Systems (This task is worth 20 Marks)

1. Discuss the differences between database and information retrieval systems in terms of their advantages and disadvantages. Find an example for database and information retrieval system to explain your arguments (10 marks).

1. Discuss the differences between data and information and their significance as a corporate resource in an OLTP environment (e.g. retailing business such as supermarket). Give one example for data and information in that environment to explain your arguments (10 marks).

Question 3– A data mining system for a bank (This task is worth 20 Marks)

A bank has been collecting data on their customers and has heard that use of data mining could increase their competitiveness. They would like you to create a brief report that explores the following.

1. What data mining is and an appropriate application for the bank (5 marks).
2. How you would go about creating the system using the data mining lifecycle below (5 marks).

• The German Credit data is useful for analysis a person’s credit for customers preference. Please access to https://archive.ics.uci.edu/ml/datasets/statlog+(german+credit+data)and http://invidio.drl.pl/files/german_credit.csv  and study the dataset.  State what are 5Cs of Credit and explain how the dataset can be applied for the 5Cs of Credit analysis. (5 marks).

1. The use of a data mining model such as a multilayer perceptron or decision tree to determine a person’s credit worthy using unseen data. Note, you will need to use a data mining tool like Google Colab / Weka to create your models and use the abovedataset to train and test these models. Please provide your analysis source codes and screen captures.  (5 marks).

Question 4: Your Big Data Idea (This task is worth 20 marks)

Identify and implement an idea that you have about how you’d use Big Data for advancedanalytics of retail chain stores such as managing customer relationship, finding new customers, and developing new products and services, finding a new trends that may facilitate understanding some scientific or engineering areas, etc.. Produce a short report the includes you have done in the following steps, your analysis results, data visualization and constructive comments / thoughts.

1. Purpose an idea and clearly outlined it (e.g. What is the purpose of your data collection and analysis). (2.5 marks).
2. Acquire the Data. You can do that in many ways including using available public large data sets. (5 marks). Description of the data sets is expected with the attributes and availability details.
3. Outline the ways to analyze the data in order to achieve the objective you set out for yourself in step 1. (2.5 marks) Using KDD with stepwise explanations would be expected. The description of data mining algorithms (e.g. clustering, association, etc.) would be expected too.
4. Description of results to be visualized (2.5 marks).
5. Constructive personal reflection (2.5 marks)
6. Proper references and in-text citations (5 marks)

Question 5: Data Retrieval (This task is worth 20 marks)

Information Retrieval (IR) includes three main processes: Indexing, Searching and Ranking.

Refer to the following diagram, perform the tasks below:

(a) build an inverted index (indexing process) (12 marks),

(b) use the Boolean Retrieval Model to find file(s) which contains the words ‘Java’ and ‘SQLDev’ (searching process) (4 marks),

(c) rank the results based on relevance. (ranking process) (4 marks)