## Contents

# HW assignment 2, ECE662 Spring 2008, Prof. Boutin

### Assignment Description

Due Tuesday April 1, 2008

Guidelines:

- Write a short report to present your results.
- Be sure to include all the relevant graphs as well as a copy of your code.
- Teamwork is encouraged, but the write up of your report must be your own.
- Please write the names of ALL your collaborators on the cover page of your report.

#### Question 1

In the Parametric Method section of the course, we learned how to draw a separation hyperplane between two classes by obtaining w0, the argmax of the cost function $ J(w)=w^TS_Bw / w^TS_ww $. The solution was found to be$ w_0= S_w^{-1}(m_1-m_2) $, where $ m_1 $ and $ m_2 $ are the sample means of each class, respectively.

Some students raised the question: can one simply use $ J(w)= w^TS_Bw $ instead (i.e. setting $ S_w $ as the identity matrix in the solution $ w_0 $? Investigate this question by numerical experimentation.

#### Question 2

Obtain a set of training data. Divide the training data into two sets. Use the first set as training data and the second set as test data.

a) Experiment with designing a classifier using the neural network approach.

b) Experiment with designing a classifier using the support vector machine approach.

c) Compare the two approaches.

Note: you may use code downloaded from the web, but if you do so, please be sure to explain what the code does in your report and give the reference.

#### Question 3

Using the same data as for question 2 (perhaps projected to one or two dimensions for better visualization),

a) Design a classifier using the Parzen window technique.

b) Design a classifier using the K-nearest neighbor technique

c) Design a classifier using the nearest neighbor technique.

d) Compare the three approaches.

### Data

- Publicly available data sources are listed here.

- Simple perl script that converts data from the libsvm to the fann format. Allows you to quickly convert data if you're using the
**FANN**and**LIBSVM**(please, follow the link under Tools_OldKiwi).

## LINK : Matlab Code

a). KNN classifiter

b). Classification using SVM

c). Demonstration of parzen window

d). Serval matlab codes realated to learning, clustering, and pattern classification

3. Tool box holding a collection of Artificial Neural Networks (ANN) algorithms implemented for Matlab

a). Neural classifier for multiple class data

b). Neural classifier for binary class data

- The voronoi diagram is related to the nearest neighbor technique.

5. The Netlab library includes software implementations of a wide range of data analysis techniques, many of which are not yet available in standard neural network simulation packages (in Matlab)

## LINK : C Code

## LINK : Documentation

2. Manual in MATLAB to accompany PatternClassification

- It contains lots of pattern recognition algorithms and gives the description and pesudo code of them.

## Links

Links to many SWM softwares, tutorials, etc: Most of these sites are compilation of several links to codes on the web

1. SVM and Kernel Methods Matlab Toolbox http://asi.insa-rouen.fr/enseignants/~arakotom/toolbox/index.html

2. SVM - Support Vector Machines Software http://www.support-vector-machines.org/SVM_soft.html

3. Some SVM sample data http://www.cs.iastate.edu/~dcaragea/SVMVis/data_sets.htm

4. LIBSVM - A library of SVM software, including both C and Matlab code. Various interfaces through several platforms available as well. http://www.csie.ntu.edu.tw/~cjlin/libsvm/

Links to Matlab Toolbox tutorials

1. SVM Matlab Bioinformatics Toolbox http://www.mathworks.com/access/helpdesk/help/toolbox/bioinfo/index.html?/access/helpdesk/help/toolbox/bioinfo/ref/svmclassify.html&http://www.mathworks.com/cgi-bin/texis/webinator/search/

2. Neural network Matlab Toolbox http://www.mathworks.com/access/helpdesk/help/toolbox/nnet/