You will get familiar with the course, the instructor, your classmates, and the learning environment. The overview of the course will also be covered during the same session.

Activities

• Reading: Syllabus
• Join NTHU ilms System
• Take Orientation Quiz before 2018/9/25

Interesting Videos

• How We Used Data to Win the Presidential Election by Dan Siroker

Understanding and reprocessing the data is the most important part in the whole data mining processes. In this session, the types of attributes and characteristics of data sets will be introduced first. Subsequently, several data preprocessing techniques are covered and followed by various similarity and distance measures. Finally, data visualization concepts will be addressed.

Related Videos

• Distance Measures of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University
• Singular Value Decomposition of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University
• Dimensionality Reduction of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University
• Dimensionality Reduction with SVD of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University
• Lecture 14 of Machine Learning (PCA parts) by Prof. Andrew Ng at Stanford University
• Lecture 15 of Machine Learning (PCA parts) by Prof. Andrew Ng at Stanford University

Activities

• Project proposal due at Sep 30
• Team up with at least one Taiwan citizenship student

In this lab session we will focus on the use of scientific computing libraries to efficiently process, transform, and manage data. Furthermore, we will provide best practices and introduce visualization tools for effectively conducting big data analysis.

Activities

• Attend the class with your personal laptop
• Assignment One should be submitted before Oct 22

Text mining is the process to extract valuable information from unstructured text. Text mining usually involves natural language processing (NLP) processes, including lexical analysis, syntactic analysis, and inference. Typical text mining tasks include text categorization, text clustering, concept/entity extraction, sentiment analysis, and entity relation modeling. We will cover some concepts in this session.

Related Videos

• Week 7: Text Mining Intro by Prof. Carolyn Rose from Carnegie Mellon University
• Week 7: Text Mining Conceptual Overview of Techniques by Prof. Carolyn Rose from Carnegie Mellon University

Time for student presentation and discussion based on one specified paper

Activities

• Team up with classmates.
• Each group prepare slides no more than 3+1 slides.
• Each group also prepare a good quiz question for your classmates

Classification, also referred as supervised learning techniques, is widely used in data mining and machine learning. The objective of classification is to predict a class from input data. In this session, various fundamental algorithms, as well as core measures, will be covered.

Related Videos

• Decision Trees of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University
• How to Construct a Tree of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University
• Information Gain of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University
• Lecture 8 of Machine Learning: support vector machines by Prof. Andrew Ng at Stanford University
• Lecture 11 of Machine Learning: Bayesian statistics by Prof. Andrew Ng at Stanford University
• Lecture 12 of Machine Learning: Expectation-maximization by Prof. Andrew Ng at Stanford University
• Lecture 13 of Machine Learning: Naive Bayes models by Prof. Andrew Ng at Stanford University

In this lab session we will focus on the use of information retrieval techniques for modeling, training, and classifying textual data. Specifically, deep learning frameworks, such as word2vec, doc2vec, and FastText, will be introduced. Additionally, traditional approaches used for text classification, such as KNN, SVM, and Naive Bayesian, will also be covered.

Activities

• Attend the class with your personal laptop
• Assignment Two should be submitted before Nov 26

Time for student presentation and discussion based on specified papers

Activities

• Each group prepare report and slides.
• Animated slides should be recorded within 5 minutes and upload to Youtube

Cluster analysis is the task to group objects, where the objects in the same cluster are more similar to each other than those in other clusters. Many clustering techniques are initially adopted in pattern recognition/signal processing domain, but now adapted in many other domains, such as data mining, machine learning, information retrieval, data compression, and computer graphics. In this session, we will introduce and discuss various clustering techniques.

Related Videos

• Lecture 12 of Machine Learning: Clustering by Prof. Andrew Ng at Stanford University
• Overview of Clustering of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University
• Hierarchical Clustering of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University
• The k Means Algorithm of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University

Time for student presentation and discussion based on specified papers

Activities

• Each group prepare report and slides.
• Animated slides should be recorded within 5 minutes and upload to Youtube

Association rule learning discovers interesting relations between variables from large data sets. By utilizing interestingness and confidence measures, it can identify strong rules discovered from data sets. It is widely used for product recommendations. In this session, several basic algorithms will be introduced and discussed.

Activities

• Reading: J. Park, M. Chen, and P. Yu. An effective hash-based algorithm for mining association rules. In SIGMOD’95
• Reading: J Han, J Pei, Y Yin, R Mao, Mining frequent patterns without candidate generation: A frequent-pattern tree approach, Data Mining and Knowledge Discovery, 2004 - Springer
• Reading: N. Pasquier, Y. Bastide, R. Taouil, and L. Lakhal. Discovering frequent closed itemsets for association rules. ICDT'99, 398-416, Jerusalem, Israel, Jan. 1999
• Reading: R. Srikant and R. Agrawal, “Mining Quantitative Association Rules in Large Relational Tables”. ACM SIGMOD96

Related Videos

• Frequent Itemsets of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University
• A Priori Algorithm of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University
• Improvements to A Priori (Advanced) of Mining of Massive Datasets by Prof. Leskovec, Rajaram, and Ullman at Stanford University

Time to evaluate. Different from other examination in our life, we do not want to assess how much we remember. It is more important to know how much we understand. Hence, each student can bring one A4-page paper with all kinds of notes into the classroom. Enjoy.

Notes

• Students can take one A4 page with them

Realizing by learning, thinking, and doing. This is the final realizing step, to develop a fine application with data mining skills, for this course. Through this practice, we not only polish our DM skills but also experience team works. With teammates, we dare to dream.

Requirements

• Students should work with several people for their project.
• Each group should generate 4 minute youtube clips to show in the class
• Each group should have a poster and a working system
• Final Project Requirement Description