---
title: "Faces"
author: "M Guerzhoy"
date: "August 3, 2016"
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

## Reading in Faces

You can skip that if you're not interested...

```{r message=F}
require(png)
require(grid)
require(glmnet)

load_actors <- function(path, names, idx, im_size){
  actors <- data.frame(matrix(nrow=length(names)*length(idx), ncol=im_size*im_size*1))
  a_ind <- 1
  names_col <- c()
  for (name in names){
    for (i in idx){
      im <- readPNG(sprintf("%s/%s%d.png", path, name, i))
      im <- rowMeans(im[,,1:3], dim=2)
      actors[a_ind,]<-im
      names_col <- c(names_col, name)
      a_ind <- a_ind + 1
    }
  }
  actors$name <- names_col
  return(actors)
}
```


Now, let's create a training and test sets

```{r}
names <- c("DanielRadcliffe", "AngieHarmon")


path <- "/media/guerzhoy/Windows/STA303/lectures/faces/cropped128"
im_size <- 128

train_set <- load_actors(path, names, 1:80, im_size)
test_set <- load_actors(path, names, 81:108, im_size)
```

Here are the first 100 dimensions from the first row of the test set.

```{r}
test_set[1,1:100]
```

There are $128\times 128 + 1 = 16385$ dimensions in total (the last one is the name):

```{r}
dim(test_set)
```

Here are the first ten names:

```{r}
test_set$name[1:10]
```

Let's display the 4-th row from the training set in a more sensible way:

```{r}
im <- data.matrix(train_set[4, 1:(im_size*im_size)])
dim(im) <- c(im_size, im_size)
grid.raster(im)
```


Now, let's fit the ridge logistic regression model. `alpha=0` means we are using ridge logistic regression. We also need to find the best parameter $\lambda$. This is done using cross-validation.

```{r message=F}
fit <- glmnet(as.matrix(train_set[,1:(im_size*im_size)]), y=factor(train_set[,(im_size*im_size)+1]), alpha=0, family="binomial")
fit_cv <- cv.glmnet(as.matrix(train_set[,1:(im_size*im_size)]), y=factor(train_set[,(im_size*im_size)+1]), alpha=0, family="binomial")
```


Looking at fit_cv, the best $\lambda$ is $3.13$, at index 97:

```{r}
fit_cv
```

Now, let's compute the predictions and see what kind of classification rate we get.

Predictions (log-odds):
```{r}
predict(fit, as.matrix(test_set[,1:(im_size*im_size)]) ,s=99)
```

Correct classification rate:

```{r}
mean((predict(fit, as.matrix(train_set[,1:(im_size*im_size)]) ,s=90)>0) == (train_set$name == "DanielRadcliffe"))

mean((predict(fit, as.matrix(test_set[,1:(im_size*im_size)]) ,s=90)>0) == (test_set$name == "DanielRadcliffe"))
```

Let's see what out model coefficients are like

```{r}
w<-as.matrix(coef(fit)[2:(im_size*im_size+1),99])
w<-(w-min(w))/(max(w)-min(w))
dim(w) <- c(im_size, im_size)
grid.raster(w)
```


Let's try the same, but setting $\lambda = 0$.

```{r}
fit0 <- glmnet(as.matrix(train_set[,1:(im_size*im_size)]), y=factor(train_set[,(im_size*im_size)+1]), alpha=0, lambda=0, family="binomial")

mean((predict(fit0, as.matrix(train_set[,1:(im_size*im_size)]) ,s=90)>0) == (train_set$name == "DanielRadcliffe"))

mean((predict(fit0, as.matrix(test_set[,1:(im_size*im_size)]) ,s=90)>0) == (test_set$name == "DanielRadcliffe"))
```

There is a lot of overfitting. Let's visualize the coefficients:

```{r}
w<-as.matrix(coef(fit0)[2:(im_size*im_size+1),1])
w<-(w-min(w))/(max(w)-min(w))
dim(w) <- c(im_size, im_size)
grid.raster(w)
```

What we see is that the model picked up random noise from the image -- it's hard to see a face at all.


Let's make the training set smaller -- that will cause overfitting even if we use a larger $\lambda$.

```{r}
train_small <- rbind(train_set[1:20,], train_set[90:110,])
fit_small <- glmnet(as.matrix(train_small[,1:(im_size*im_size)]), y=factor(train_small[,(im_size*im_size)+1]), alpha=0,  family="binomial")

fit_cv <- cv.glmnet(as.matrix(train_small[,1:(im_size*im_size)]), y=factor(train_small[,(im_size*im_size)+1]), alpha=0, family="binomial")

mean((predict(fit_small, as.matrix(train_small[,1:(im_size*im_size)]) ,s=90)>0) == (train_small$name == "DanielRadcliffe"))

mean((predict(fit0, as.matrix(test_set[,1:(im_size*im_size)]) ,s=90)>0) == (test_set$name == "DanielRadcliffe"))

```

Let's visualize the coefficients from the small model.

```{r}
w<-as.matrix(coef(fit_small)[2:(im_size*im_size+1),1])
w<-(w-min(w))/(max(w)-min(w))
dim(w) <- c(im_size, im_size)
grid.raster(w)
```

It's basically the average of a few pictures -- you can even see the glasses.

