Python Pandas

Introduction
Python pandas

Open Source
Python Library

Simple yet Powerful

and Expressive Tool

Data Manipulation
and Analysis
Application of Pandas

Application of Pandas

Natural Language
Processing Statistics Analytics Big Data

Recommendation
Engine
Stock Prediction Data Science
Pandas Vs. Numpy

Numpy Pandas

Low level data structure. (np.array) High level data structures. (data frame)
It provides in-memory 2d table object
called data frame.

Support for large multidimensional arrays More streamlined handling of tabular data,
and matrices. and rich time series functionality.

A wide range of mathematical array Data alignment, handling missing data,

operations. groupby, merge, and, join methods.
Installation

Open terminal program (for Mac user) or command line (for Windows) and install it
using following command:

conda install pandas

Or

pip install pandas

Installation

● Alternatively, you can install pandas in a jupyter notebook using below code:

!pip install pandas

● To import pandas we import it with a shorter name:

import pandas as pd
Components of the pandas

● Series and dataframe are two primary components of the pandas

● A series is typically a column, and a data frame is a multi-dimensional table
made up of a group of Series
One Series
Dimensional

Components of the
Pandas

DataFrame

Multi
Dimensional

Panel Data
Pandas Series
Pandas series

Series can be created using the following constructor:

To copy the data

It takes various forms It is for data type

like ndarray, lists

Values must be unique and the

same length as data
Creating a Series
Pandas series

A pandas series can be created out of a python list or numpy array

Using list Using numpy array

Pandas series

We can create our index values while creating a series

Pass index parameter

Pandas series

String as a row index

Pandas series

● Create a series from python dictionary

● The key becomes the row index while the value becomes the value at that row
index
Pandas series

Here, the list items remain part of a single row index

Pandas series

To display the index names and values of the series use “.index” and “.values”
respectively.
Accessing a Series
Accessing elements of series

Use the index operator [] to access element in a series

Retrieve first five

elements
Accessing elements of series

Retrieve last five elements

Access element using index

Use index element to access

element
Access element using index

Use index element to access

element
Access element using index

Retrieve multiple elements

using a list of index
Filtering a Series
Filter the values

Filter all the values that are

greater than 15
Arithmetic Operations
Multiply each element in the series by 2

Use ‘*’ operator to perform

multiplication
Add corresponding elements of two series

Use ‘+’ operator to perform

addition
Ranking and Sorting
Ranking in the series

It returns the rank of the

underlying data
Sort series in ascending order
Sort series in descending order
Sort series based on index
Check for Null Values
Check null values using .isnull()

True indicates that the value is null

Check null values using .notnull()

False indicates that the value is null

Pandas DataFrame
Pandas dataframe

A data frame is two dimensional data structure, i.e., data aligned in tabular
manner(rows and column)

Features of DataFrame:

Potentially Columns are of Different Types

Size - Mutable

Labeled axes(rows and column)

Can Perform Arithmetic Operations on Rows and Column

Reading data from different
sources
Reading data from csv file

Use ‘read_csv()’ function from pandas to read data from csv file
Reading data from xlsx file

Use ‘read_excel()’ function from pandas to read data from xlsx file
Reading data from zip file

Read the zip file

Open csv file

Read the csv file

Reading data from text file

Use ‘read_csv()’ function from pandas to read data from text file
Reading data from json file

Use ‘read_json()’ function from pandas to read data from json file
Reading data from xml file
Import package to read xml
file

Parse or extract the xml file

Assign the column names of

output dataframe

Use for loop to extract all the

data

Append each observation in

data to ‘rows’

Create a dataframe ‘xml_df’

Reading data from html file

Use ‘read_html()’ function from pandas to read data from html file
Pandas DataFrame
Pandas DataFrame

● Using the previous mentioned ways to import the data in python, the data is
always a python DataFrame

● Let us now see some operations and manipulations on DataFrames

Creating DataFrames
Creating data frame using single list

1. First create a list.

2. Convert it into a
DataFrame.
Creating dataframe using list of list
Creating dataframe from dictionary of ndarrays
Creating data frame using arrays

Create list of index

Creating data frame using list of dictionaries
Read first five rows of the data

DataFrame.head() will display first five rows of the data

Read last five rows of the data

DataFrame.tail() will display last five rows of the data

Shape of DataFrame
Know more about data

● Check the dimension of the data

● Check the data type

Know more about data

● Use “DataFrame.info()” to know get

information on shape of the data, the data
type and null values in each variable

● Here we see ‘df_market” has 3 variables

with 25 observations in each

● These are non-null observations

● There are 2 categorical variables and one

numeric variable.
Indexing DataFrames
Dealing with rows and column

● Indexing is frequently required in DataFrame. It may serve the purpose

of cross tables or pivot tables

● We can either use the .iloc[] function, the .loc[] function or use some
conditions.

● The “.iloc[]” allows us to retrieve rows and columns by position, and

The “.loc[]” allows us to specify the column name or index to subset.
Dealing with rows and column

Example: Create a new DataFrame as show and access the value that is at index 0 in column
‘Name’
Dealing with rows and column

Select row by iloc[] method

Dealing with rows and column

Select 4th and 6th rows

Dealing with rows and column

Select first three columns by using column number

Dealing with rows and column

Select first and third column

Dealing with rows and column

● loc[] function selects data by the label of the rows and column
● Access the value that is at index 1 in column ‘Score’ using loc method
Dealing with rows and column

Select multiple value by row label and column label using loc
Dealing with rows and column

Select two columns from the data frame

Conditional Subsetting

Subset students who have marks more than 12.

Conditional Subsetting

Subset students who either have more than two attempts or qualify the exam.
Sorting DataFrames
Sort data frame

Sort data frame based on the values of the column

Sort data frame

Sort data frame based on the values of the column in descending order
Sort data frame

Sort data frame based on the values of the multiple columns

Sort data frame

● Note that, while sorting dataframe by multiple columns, pandas sort_value() sorts
the first variable and then the next variable next

● In this case, the function first

sorted the variable ‘percentage’
and then the variable ‘store’
Sort data frame

To sort the index of

the DataFrame use
“.sort_index()”
Ranking DataFrames
Rank the data frame

Rank the dataframe in pandas on ascending order

Rank the data frame

Rank the dataframe in pandas on descending order

Rank the data frame

● Rank the dataframe in pandas by minimum value of the rank

● Rank the data frame in descending order of percentage and if found two
percentage are same then assign the minimum rank to both the percentage
Rank the data frame

● Rank the dataframe in pandas by maximum value of the rank

● Rank the data frame in descending order of percentage and if found two
percentage are same then assign the maximum rank to both the percentage
Rank the data frame

● Rank the dataframe in pandas by dense rank

● Rank the data frame in descending order of score and if found two scores are same
then assign the same rank . Dense rank does not skip any rank (in min and max
ranks are skipped)
Thank You