[SOLVED] Is there a quicker method for iterating over rows in Python to calculate a feature?

Issue

I have a Pandas Dataframe df that details Names of players that play a game. The Dataframe has 2 columns of ‘Date’ they played a game and their name, sorted by Date.

What I am trying to accomplish is to time-efficiently calculate the previous number of games each player has played before they play the upcoming game that day.

For the example Dataframe above, calculating the players previous number of games would start at 0 and look like follows.

I have tried the following codes and although they have delivered the correct result, they took many days for my computer to run.

Attempt 1:

for i in range(0, len(df) ):
   df['Previous Games'][i] = len( df[ (df['Name'] == df['Name'][i]) & (df['Date'] < df['Date'][i]) ] )

Attempt 2:

df['Previous Games'] = [ len( df[ (df['Name'] == df['Name'][i]) & (df['Date'] < df['Date'][i]) ] ) for i in range(0, len(df) ) ]

Although Attempt 2 was slightly quicker, it was still not time-efficient so I need help in finding a faster method.

Solution

Any time you write "for" and "pandas" anywhere close together you are probably doing something wrong.

It seems to me you want the cumulative count:

df["prev_games"] = df.sort_values('Date').groupby('Name').cumcount()

Answered By – Christian Sloper

Answer Checked By – Katrina (BugsFixing Volunteer)