任何分组(groupby)操作都涉及原始对象的以下操作之一。它们是：
- 分割对象
- 应用一个函数
- 结合的结果
在许多情况下，将数据分成多个集合，并在每个子集上应用一些函数。在应用函数中，可以执行以下操作：
- 聚合 - 计算汇总统计量
- 转换 - 执行一些特定于组的操作
- 过滤 - 在某些情况下丢弃数据
下面来创建一个DataFrame对象并对其执行所有操作：
In [2]:
import pandas as pd
In [3]:
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
In [4]:
df = pd.DataFrame(ipl_data)
In [5]:
print (df)
Team Rank Year Points 0 Riders 1 2014 876 1 Riders 2 2015 789 2 Devils 2 2014 863 3 Devils 3 2015 673 4 Kings 3 2014 741 5 kings 4 2015 812 6 Kings 1 2016 756 7 Kings 1 2017 788 8 Riders 2 2016 694 9 Royals 4 2014 701 10 Royals 1 2015 804 11 Riders 2 2017 690
In [6]:
import pandas as pd
In [7]:
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
In [8]:
df = pd.DataFrame(ipl_data)
In [9]:
print (df.groupby('Team'))
<pandas.core.groupby.generic.DataFrameGroupBy object at 0x7fa8bc4477d0>
In [10]:
import pandas as pd
In [11]:
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017], 'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
In [12]:
df = pd.DataFrame(ipl_data)
In [13]:
print (df.groupby('Team').groups)
{'Devils': [2, 3], 'Kings': [4, 6, 7], 'Riders': [0, 1, 8, 11], 'Royals': [9, 10], 'kings': [5]}
按多列分组：
In [14]:
import pandas as pd
In [15]:
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
In [16]:
df = pd.DataFrame(ipl_data)
In [17]:
print (df.groupby(['Team','Year']).groups)
{('Devils', 2014): [2], ('Devils', 2015): [3], ('Kings', 2014): [4], ('Kings', 2016): [6], ('Kings', 2017): [7], ('Riders', 2014): [0], ('Riders', 2015): [1], ('Riders', 2016): [8], ('Riders', 2017): [11], ('Royals', 2014): [9], ('Royals', 2015): [10], ('kings', 2015): [5]}
In [18]:
import pandas as pd
In [19]:
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
In [20]:
df = pd.DataFrame(ipl_data)
In [21]:
grouped = df.groupby('Year')
In [22]:
for name,group in grouped:
print (name)
print (group)
2014
Team Rank Year Points
0 Riders 1 2014 876
2 Devils 2 2014 863
4 Kings 3 2014 741
9 Royals 4 2014 701
2015
Team Rank Year Points
1 Riders 2 2015 789
3 Devils 3 2015 673
5 kings 4 2015 812
10 Royals 1 2015 804
2016
Team Rank Year Points
6 Kings 1 2016 756
8 Riders 2 2016 694
2017
Team Rank Year Points
7 Kings 1 2017 788
11 Riders 2 2017 690
默认情况下，groupby 对象具有与分组名相同的标签名称。
In [23]:
import pandas as pd
In [24]:
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
In [25]:
df = pd.DataFrame(ipl_data)
In [26]:
grouped = df.groupby('Year')
In [27]:
print (grouped.get_group(2014))
Team Rank Year Points 0 Riders 1 2014 876 2 Devils 2 2014 863 4 Kings 3 2014 741 9 Royals 4 2014 701
In [28]:
import pandas as pd
import numpy as np
In [29]:
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
In [30]:
df = pd.DataFrame(ipl_data)
In [31]:
grouped = df.groupby('Year')
In [32]:
print (grouped['Points'].agg(np.mean))
Year 2014 795.25 2015 769.50 2016 725.00 2017 739.00 Name: Points, dtype: float64
/tmp/ipykernel_1578/445005377.py:1: FutureWarning: The provided callable <function mean at 0x7fa9082fdda0> is currently using SeriesGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead. print (grouped['Points'].agg(np.mean))
另一种查看每个分组的大小的方法是应用 size() 函数：
In [33]:
import pandas as pd
import numpy as np
In [34]:
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
In [35]:
df = pd.DataFrame(ipl_data)
In [36]:
grouped = df.groupby('Team')
In [37]:
print (grouped.agg(np.size))
Rank Year Points Team Devils 2 2 2 Kings 3 3 3 Riders 4 4 4 Royals 2 2 2 kings 1 1 1
In [38]:
import pandas as pd
import numpy as np
In [39]:
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
In [40]:
df = pd.DataFrame(ipl_data)
In [41]:
grouped = df.groupby('Team')
agg = grouped['Points'].agg([np.sum, np.mean, np.std])
/tmp/ipykernel_1578/1839800803.py:2: FutureWarning: The provided callable <function sum at 0x7fa9082fc9a0> is currently using SeriesGroupBy.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "sum" instead. agg = grouped['Points'].agg([np.sum, np.mean, np.std]) /tmp/ipykernel_1578/1839800803.py:2: FutureWarning: The provided callable <function mean at 0x7fa9082fdda0> is currently using SeriesGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead. agg = grouped['Points'].agg([np.sum, np.mean, np.std]) /tmp/ipykernel_1578/1839800803.py:2: FutureWarning: The provided callable <function std at 0x7fa9082fdee0> is currently using SeriesGroupBy.std. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "std" instead. agg = grouped['Points'].agg([np.sum, np.mean, np.std])
In [42]:
print (agg)
sum mean std Team Devils 1536 768.000000 134.350288 Kings 2285 761.666667 24.006943 Riders 3049 762.250000 88.567771 Royals 1505 752.500000 72.831998 kings 812 812.000000 NaN
In [43]:
import pandas as pd
import numpy as np
In [44]:
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
In [45]:
df = pd.DataFrame(ipl_data)
In [46]:
grouped = df.groupby('Team')
score = lambda x: (x - x.mean()) / x.std()*10
In [47]:
print (grouped.transform(score))
Rank Year Points 0 -15.000000 -11.618950 12.843272 1 5.000000 -3.872983 3.020286 2 -7.071068 -7.071068 7.071068 3 7.071068 7.071068 -7.071068 4 11.547005 -10.910895 -8.608621 5 NaN NaN NaN 6 -5.773503 2.182179 -2.360428 7 -5.773503 8.728716 10.969049 8 5.000000 3.872983 -7.705963 9 7.071068 -7.071068 -7.071068 10 -7.071068 7.071068 7.071068 11 5.000000 11.618950 -8.157595
In [48]:
import pandas as pd
import numpy as np
In [49]:
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
In [50]:
df = pd.DataFrame(ipl_data)
filter = df.groupby('Team').filter(lambda x: len(x) >= 3)
In [51]:
print (filter)
Team Rank Year Points 0 Riders 1 2014 876 1 Riders 2 2015 789 4 Kings 3 2014 741 6 Kings 1 2016 756 7 Kings 1 2017 788 8 Riders 2 2016 694 11 Riders 2 2017 690
在上述过滤条件中，要求返回参加过 3 次或以上 IPL 的团队。