Pandasdrilldescribe
df.describe() returns count, mean, std, min, 25%, 50%, 75% and max for every numeric column. include="all" adds text columns (count, unique, top, freq), percentiles=[...] changes the quantiles, and .T flips the table so each column is a row.
Syntax
df.describe(percentiles=None, include=None, exclude=None)| Part | What it is |
|---|---|
percentiles | List of fractions between 0 and 1, e.g. [.1, .5, .9] |
include | "all", or a list of dtypes such as ["object"] to describe text columns |
Examples
import pandas as pd
df = pd.DataFrame({"dept": ["Sales", "IT", "Sales", "HR"],
"salary": [50000, 62000, 58000, 45000]})
df.describe()Result: salary count 4.000000 mean 53750.000000 std 7675.719293 min 45000.000000 25% 48750.000000 50% 54000.000000 75% 59000.000000 max 62000.000000
df.describe(include="object")Result: dept count 4 unique 3 top Sales freq 2
df["salary"].describe(percentiles=[.1, .5, .9]).round(0)Result: count 4.0 mean 53750.0 std 7676.0 min 45000.0 10% 46500.0 50% 54000.0 90% 60800.0 max 62000.0 Name: salary, dtype: float64
Common mistakes
- Text columns are missing: by default only numeric columns are described; pass include="all" or include="object".
- Numbers in scientific notation: set pd.options.display.float_format = "{:,.2f}".format or round the result.
- std looks different from Excel's STDEV.P: pandas uses the sample standard deviation (ddof=1), like STDEV.S.
Practice describe
These drills check your answer on live data, free, no account needed.
- Several statistics at onceFor each product, return the minimum, maximum and mean revenue per order. The columns should be `min`, `max` and `mean`, in that order.
Questions
How do I describe by group?
df.groupby("dept")["salary"].describe() gives the same statistics for each group as rows.
How do I get just the mean and median?
df.agg(["mean", "median"]) or df.median(numeric_only=True); describe is for the full overview.
What does 50% mean in describe?
It is the median: half the values are at or below it.
Related
- aggHow to use agg in pandas to compute several statistics per group, different functions per column, named aggregations with flat column names, and lambdas.
- groupbyHow groupby works in pandas: group by one or more columns, apply sum, mean, count or size, get a flat table with as_index=False, and iterate over groups.
- value_countsHow to count values in a pandas column with value_counts: counts sorted by frequency, percentages with normalize=True, including NaN, and two-column combos.
- unique and nuniqueHow to get distinct values in pandas with unique, count them with nunique, count distinct per group with groupby, handle NaN, and see counts for all columns.