# Lecture 23: Logistic Regression • machine learning
• when labeled you have supervised learning
• when quantitative do regression
• when categorical do classification
• when unlabeled you have unsupervised learning
• dimensionality reduction
• clustering
• finally reinforcement learning

## Kinds of Classification • binary two classes
• multiclass [cat, dog, car]
• structured prediction ? • try least squares regression • two classes • truncated least square

## Estimating the chance of success  • two difference coins case by case • single expression $$p^y(1-p)^{1-y}$$ • estimate probability find value that maximize the function
• take a log • as a sum • you can minimize the average? • what is the function in the two hard cases? • as a loss function there is a penalty

## Logistic Function

• linear functions are not good for probabilities • t can be infinity and negative infinity
• take e of both sides • model probability on the real line
• sigmoid, defined on whole line, smooth, increasing, elongated S • derivative

# Log Odds as a Linear Function

• features
• linear combo of features • log odds
• probabiligty is of the log odds ## The Steps of the Model

• generalized linear model • linear regression continuous
• categorical (probability Y is 1) • increase x by one unit  • linearly seperable data • you need a little bit of uncertainty • with regularization term

## Logistic Loss Function    ## Logistic Regression in Scikit Learn ## Log Loss • or cross-entropy loss • log loss is convex!