Scribd
Upload
10 views

Logistic Regression Example (1)

The document explains how Logistic Regression is used for sentiment analysis through a step-by-step approach, starting from binary classification to making predictions. It details the process of converting text into numerical representations, assigning weights to words, calculating scores, applying the sigmoid function, and making final predictions. Additionally, it provides a Python implementation for training a Logistic Regression model on sentiment data, including data cleaning, feature extraction, and model evaluation.

Uploaded by

Asil Zulfiqar 4459-FBAS/BSCS4/F21
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
10 views

Logistic Regression Example (1)

The document explains how Logistic Regression is used for sentiment analysis through a step-by-step approach, starting from binary classification to making predictions. It details the process of converting text into numerical representations, assigning weights to words, calculating scores, applying the sigmoid function, and making final predictions. Additionally, it provides a Python implementation for training a Logistic Regression model on sentiment data, including data cleaning, feature extraction, and model evaluation.

Uploaded by

Asil Zulfiqar 4459-FBAS/BSCS4/F21
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 7

How Logistic Regression Assigns Scores in Sentiment Analysis (Step-by-Step)

Alright, let’s break it down like we're explaining to a beginner. We’ll go step by step to
understand how Logistic Regression assigns scores and makes predictions.

🔹 Step 1: Understanding the Core Idea


Logistic Regression is used for binary classification, meaning it decides between two
categories (e.g., positive vs. negative sentiment).

Goal:

 Each word in a review contributes to the final score.


 The score is converted into a probability using the sigmoid function.
 If the probability is greater than 0.5 → Positive (1).
 Otherwise, Negative (0).

🔹 Step 2: Representing Text as Numbers


Before the model can work, we must convert text into numbers.

Example Reviews:

1️⃣ "I love this movie" → Positive (1)


2️⃣ "I hate this movie" → Negative (0)

Bag of Words (BoW) Representation:

I love hate this movie


Review 1 1 1 0 1 1
Review 2 1 0 1 1 1

Each review is now represented as a vector of numbers, which Logistic Regression can
process.

🔹 Step 3: Assigning Weights to Words


Logistic Regression Equation:

score=w1⋅x1+w2⋅x2+...+wn⋅xn+b\text{score} = w_1 \cdot x_1 + w_2 \cdot x_2 + ... + w_n \


cdot x_n + b
Where:

 wiw_i → Weight assigned to each word (how important it is)


 xix_i → Word presence (1 if the word appears, 0 if it doesn’t)
 bb → Bias term (a constant value)

📌 Example Weights (learned during training):

Word Weight (ww)


I 0.1
love 2.5
hate -3.0
this 0.5
movie 1.0

📌 Bias term: b=−0.5b = -0.5

🔹 Step 4: Calculating the Score


Now, let’s compute the score for each review using the weights.

Review 1: "I love this movie"

score=(0.1×1)+(2.5×1)+(−3.0×0)+(0.5×1)+(1.0×1)+(−0.5)\text{score} = (0.1 \times 1) +


(2.5 \times 1) + (-3.0 \times 0) + (0.5 \times 1) + (1.0 \times 1) + (-0.5)
=0.1+2.5+0+0.5+1.0−0.5= 0.1 + 2.5 + 0 + 0.5 + 1.0 - 0.5 =3.6= 3.6

Review 2: "I hate this movie"

score=(0.1×1)+(2.5×0)+(−3.0×1)+(0.5×1)+(1.0×1)+(−0.5)\text{score} = (0.1 \times 1) +


(2.5 \times 0) + (-3.0 \times 1) + (0.5 \times 1) + (1.0 \times 1) + (-0.5)
=0.1+0−3.0+0.5+1.0−0.5= 0.1 + 0 - 3.0 + 0.5 + 1.0 - 0.5 =−1.9= -1.9

🔹 Step 5: Applying the Sigmoid Function


The score is now converted into a probability using the sigmoid function:

P=11+e−scoreP = \frac{1}{1 + e^{-\text{score}}}

📌 Why do we use sigmoid?

 It squashes the score into a range between 0 and 1.


 If P > 0.5 → Positive sentiment (1)
 Otherwise, Negative sentiment (0)
Review 1: "I love this movie" (Score = 3.6)

P=11+e−3.6P = \frac{1}{1 + e^{-3.6}} P≈11+0.027=0.974P \approx \frac{1}{1 + 0.027} =


0.974

🔹 Probability = 97.4% → Positive (1) ✅

Review 2: "I hate this movie" (Score = -1.9)

P=11+e1.9P = \frac{1}{1 + e^{1.9}} P≈11+6.69=0.13P \approx \frac{1}{1 + 6.69} = 0.13

🔹 Probability = 13% → Negative (0) ❌

🔹 Step 6: Making the Final Prediction


Review Score Sigmoid Output Prediction
"I love this movie" 3.6 0.974 Positive (1) ✅
"I hate this movie" -1.9 0.13 Negative (0) ❌

🎯 The model successfully classified the reviews! 🎯

🔹 Step 7: Training Logistic Regression


Before making predictions, the model learns the weights from data.

How does the model learn?

1. Starts with random weights 📊


2. Calculates predictions using sigmoid
3. Compares predictions with actual labels (positive/negative)
4. Adjusts weights using an optimization method (Gradient Descent)
5. Repeats the process until predictions improve 📈

🔹 Summary of the Process 🚀


Step What Happens? Example
1️⃣ Convert Text to Represent words as "love" = 1, "hate" = 1
Numbers numbers
2️⃣ Assign Weights Learn word importance "love" → +2.5, "hate" → -3.0
Multiply words by Score = (2.5 × 1) + (-3.0 × 0) +
3️⃣ Compute Score ...
weights
Step What Happens? Example
Convert score to
4️⃣ Apply Sigmoid Sigmoid(3.6) → 0.974
probability
5️⃣ Make Prediction If P > 0.5 → Positive 0.974 → Positive (1)

🔹 Why Use Logistic Regression for Sentiment Analysis?


✅ Simple → Easy to implement
✅ Fast → Works well on small datasets
✅ Interpretable → You can see which words affect sentiment

🚀 Now you understand how Logistic Regression assigns scores step by step! 🎯

Alright! Let’s break down logistic regression for sentiment analysis step by step, assuming
you’re a complete beginner. 🚀

1. What is Sentiment Analysis?


Sentiment Analysis is a type of text classification where we determine whether a given text
expresses positive, negative, or sometimes neutral sentiment.

For example:
✅ "I love this movie!" → Positive
❌ "This product is terrible!" → Negative
😐 "The service was okay." → Neutral

2. Why Use Logistic Regression?


Logistic Regression is a simple yet powerful algorithm used for binary classification (two
categories). Since sentiment analysis is often a positive vs. negative task, logistic regression
works well.

It predicts probabilities using the sigmoid function, which ensures outputs are between 0 and
1. If the probability is above 0.5, we classify it as positive; otherwise, it’s negative.

3. Steps to Perform Sentiment Analysis Using Logistic


Regression
Let’s implement it in Python! 🐍

Step 1: Import Libraries


import numpy as np
import pandas as pd
import re # For text cleaning
import nltk # Natural Language Processing tools
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize
from sklearn.feature_extraction.text import CountVectorizer,
TfidfVectorizer
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report

Note: Run nltk.download('stopwords') and nltk.download('punkt') if you’re using


nltk for the first time.

Step 2: Load Dataset

For simplicity, let’s assume we have a dataset with two columns:

 "text" (contains reviews)


 "sentiment" (1 = positive, 0 = negative)

# Sample dataset (usually loaded from CSV)


data = pd.DataFrame({
'text': [
"I love this movie, it's amazing!",
"This product is the worst. Never buying again.",
"Absolutely fantastic experience!",
"Terrible customer service. Not recommended.",
"I'm so happy with my purchase!",
"Worst experience ever."
],
'sentiment': [1, 0, 1, 0, 1, 0] # 1 = positive, 0 = negative
})

Step 3: Clean the Text Data

Text data is messy! We need to:

 Remove special characters, numbers, and punctuation


 Convert everything to lowercase
 Remove stopwords (common words like the, is, and)
 Tokenize (split text into words)

nltk.download('stopwords')
nltk.download('punkt')
stop_words = set(stopwords.words('english'))
def clean_text(text):
text = text.lower() # Convert to lowercase
text = re.sub(r'\W', ' ', text) # Remove non-word characters
text = re.sub(r'\s+', ' ', text).strip() # Remove extra spaces
words = word_tokenize(text) # Tokenization
words = [word for word in words if word not in stop_words] # Remove
stopwords
return " ".join(words)

data['clean_text'] = data['text'].apply(clean_text)
print(data[['text', 'clean_text']])

Step 4: Convert Text to Numbers (Feature Extraction)

Since machine learning models only understand numbers, we use TF-IDF or Bag of Words
to convert text into numerical features.

vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(data['clean_text']) # Convert text into
numerical features
y = data['sentiment'] # Target labels (0 or 1)

Step 5: Train-Test Split

We split our data into training and testing sets (80%-20%).

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,


random_state=42)

Step 6: Train Logistic Regression Model

Now, we train the model using LogisticRegression.

model = LogisticRegression()
model.fit(X_train, y_train)

Step 7: Make Predictions

We test our model on unseen data.

y_pred = model.predict(X_test)

Step 8: Evaluate the Model

We check accuracy and performance.

print("Accuracy:", accuracy_score(y_test, y_pred))


print("Classification Report:\n", classification_report(y_test, y_pred))

9. Make a Custom Prediction


Let’s check the sentiment of a new review.

def predict_sentiment(review):
clean_review = clean_text(review) # Clean the text
vectorized_review = vectorizer.transform([clean_review]) # Convert to
numerical features
prediction = model.predict(vectorized_review) # Predict sentiment
return "Positive" if prediction[0] == 1 else "Negative"

print(predict_sentiment("I really love this product, it's fantastic!")) #


Positive
print(predict_sentiment("This is the worst thing ever.")) # Negative

Summary of Key Steps


1. Load dataset 📊
2. Clean the text ✨
3. Convert text to numbers 🔢
4. Split into training & testing sets 🏋️‍♂️
5. Train logistic regression model 🤖
6. Test & evaluate performance 📈
7. Predict new sentiments 🎯

Next Steps?
 Try using a larger dataset (like IMDB movie reviews).
 Experiment with different feature extraction methods (CountVectorizer, Word
Embeddings).
 Tune hyperparameters (change solver, C values in LogisticRegression()).
 Explore deep learning models (LSTMs, Transformers) for even better accuracy!

Would you like me to help with dataset sources or improvements? 🚀

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy