tags: #python/data_science/eda

Basic syntax

To check for duplicate rows:

df.isnull()

This returns a DataFrame of the same shape as the input object, indicating if each element is NaN (missing or null) (True) or not (False).

Retrieve Total Number of Missing Values

df.isnull().sum()

This returns the count of null values for each column in the DataFrame:

A    1
B    1
C    1
dtype: int64

To Check Whether There are Missing Values in the Overall Dataset

df.isnull().any()

This returns a boolean series indicating whether any null value exists in each column of the DataFrame:

A     True
B     True
C     True
dtype: bool

Getting Percentage of Missing Values for Each Column

(data.isnull().sum()/(len(df)))*100