YT YouTube2Text

YouTube2Text → Transcripts → Learn Pandas in 1 hour! 🐼

Learn Pandas in 1 hour! 🐼 — Transcript

by Bro Code · 8,014 words · 1,289 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00Hey everybody. In this video, I'm going
0:01to teach you everything you need to know
0:03to start working with pandas. So, I
0:05encourage you to sit back, relax, and
0:08enjoy the show.
0:11Pandas is a Python library built on top
0:14of NumPy. It's widely used in data
0:16analysis, data science, and machine
0:18learning. The name pandas comes from the
0:20term panel data. Unfortunately, it has
0:22nothing to do with pandas, the animal.
0:24Using this library, you typically work
0:26with objects known as series and dataf
0:29frames. A series is like a
0:31one-dimensional labeled column. A dataf
0:33frame is like a two-dimensional labeled
0:35grid or table. With this library, you
0:38can import, display, manipulate, and
0:40export tabular data. Think of pandas as
0:43Python's version of Microsoft Excel, but
0:45on steroids. Before getting started with
0:48the series, I would recommend learning
0:50everything up to Python lists. I do have
0:52a free 1-hour course on YouTube to catch
0:54you up to speed if you're interested.
0:57Here's how we can start working with
0:58pandas. Within your IDE at the top,
1:02we're going to import the library of
1:05pandas, then give it an alias, as pd,
1:09meaning pandas. You'll likely need to
1:12download this package. And honestly, the
1:14easiest way to do that, if pandas is
1:16underlined red, you can just simply
1:19install this package. That should clear
1:21up that warning. Otherwise, what you
1:24could do is go to your left toolbar and
1:26open up a terminal. You can also use the
1:29shortcut of Alt + F12. You can use pip
1:32to install pandas.
1:37If you have multiple versions of Python
1:39downloaded, you'll need to use a
1:40different command. If that doesn't work,
1:43python-m
1:46then pip install pandas. One of those
1:49should work for you. Then hit enter.
1:54To check the current version of pandas,
1:56we can print
1:58access pandas as pd double underscore
2:02also known as dunder version double
2:06underscore again. And the current
2:08version of pandas that I'm running is
2:112.3.2.
2:13All right. And we are ready for our
2:15first lesson.
2:22All right, everybody. To kick off this
2:24Panda series, I got to talk about well,
2:26series. A series is a panda's
2:29one-dimensional labeled array. Think of
2:31it like a single column in a
2:33spreadsheet. It's one-dimensional. A
2:36good analogy would be a single hallway
2:38with apartments. Each apartment holds
2:41some data. Each apartment also has some
2:43sort of label or number. A series is a
2:46one-dimensional labeled array. Here's
2:48how we can create one. We'll start with
2:51a Python list. Let's say we have some
2:54data. Then we'll add a few numbers. 100
2:57102 104.
3:01I can convert this list to be a series.
3:04Here's how. I will create a series
3:06object equals access pandas spd call the
3:12series constructor.
3:14Now this is a constructor not a
3:16function. The s is uppercase.
3:19As an argument we will pass in our data.
3:23And then if I were to print my series
3:24object
3:26here's what it would say.
3:30There's a couple things going on here.
3:32All of our data is arranged in a single
3:35column. 100 102 104. The left hand side
3:39has an index. The index has labels
3:42starting with zero 0 1 2 and on the
3:46bottom we have metadata about this
3:48series. The data type of what we're
3:51storing are 64bit integers
3:54100 102 104. If we were to change the
3:57data type the metadata is going to
3:59change too. I'll add some decimal
4:02portions.1
4:042.3
4:07and let's see what changes.
4:10Now the data type is 64bit floatingoint
4:13numbers or we could use letters or
4:16strings
4:18A
4:21B
4:24C.
4:26Now the data type of what we're storing
4:28are objects
4:31or even booleans
4:33true
4:35false
4:37true.
4:40The data type of this series is bool
4:42meaning boolean true or false. But let's
4:46change these back to be numbers. 100 102
4:50104.
4:53Now when you construct a series object,
4:56you can set an index with custom labels.
4:59The default behavior for each label is
5:01to start at zero and increment by one.
5:04But we can pass in something custom.
5:05This will be another argument within our
5:07series constructor. We'll pass in a
5:09keyword argument for readability
5:12index equals. We can pass in a list, a
5:16tpple, a dictionary, a numpy array, or
5:19even another series. To keep it simple,
5:22let's set the first label to be A, then
5:25B, then C.
5:28Here we go. Here are the new labels.
5:31With our index, we have new labels, A B
5:35C, but the values are still the same,
5:37100, 102, 104. You could get creative
5:40with your labels. For example, going
5:42back to our apartment analogy, let's say
5:45apartment
5:46number one, then apartment number two,
5:52then apartment number three.
5:55We have apartment one is 100, apartment
5:58number two is 102, apartment number
6:01three is 104.
6:06We'll change that back. If at any time
6:09you need to access a value directly
6:10within a series, you'll take your
6:12series, access the lock property. Lock
6:16means location by label, but people just
6:19say lock.
6:21Use the subscript operator which is a
6:23set of straight brackets. Then you will
6:26select a label for example a. Return the
6:30value where the label is a.
6:34This will return a value whatever is at
6:36this label. A is 100,
6:40B is 102,
6:44C is 104. If there's a label that
6:46doesn't exist, like D, we get a key
6:49error. Let's change that back.
6:53To update one of these values, we'll
6:55have to access that lock property.
6:58Take our series,
7:00access the lock property within the
7:02subscript operator, locate the value at
7:06label C.
7:08Then let's set this to be 200.
7:12Then we will print our series.
7:17C is now 200. So to print or access a
7:20value, you access the lock property.
7:23We're locating a value by label. Now you
7:26can also locate by integer position.
7:30That's iO series dot the letter I lock.
7:36If I need the first value, the integer
7:38location is going to be zero.
7:41That returns 100,
7:441 is 102,
7:47and 2 is 104. It's pretty similar to
7:50accessing an array in other programming
7:52languages. So you got eyelock for
7:54integer location or lock location by
7:57label.
7:59Okay. Now, we're going to filter by
8:01value. We'll need a few more values.
8:04Let's say we have 200 and 202. We'll
8:08need a few more labels though.
8:10Currently, there's a mismatch. Value
8:12error. Length of values five does not
8:14match the length of index 3. We'll add a
8:17few more.
8:19We have a b c d e.
8:25There we go. Let's filter by values. We
8:28will return any values that are 200 or
8:31greater. Here's how.
8:33take our series, use the subscript
8:36operator, and here we'll write a
8:38condition. Return any values within our
8:42series that are greater than or equal to
8:46200.
8:48This will print
8:50the label of D has a value of 200. The
8:53label of E has a value of 2002.
8:57Let's switch this to be less than 200.
9:02A is 100, B is 102, C is 104.
9:07All right, here's a more complex
9:09example. Rather than using a Python list
9:11for this next example, we'll be using a
9:13dictionary.
9:17As you guys might know, I really like
9:19pizza. And well, I need to go on a diet
9:22because I've eaten too much pizza. I'm
9:24going to use a Python dictionary to
9:26record how many calories I've eaten per
9:28day. I'll create a dictionary. We'll
9:30name it calories. Calories equals a
9:34Python dictionary. A dictionary is made
9:36of key value pairs.
9:39For the key of day one,
9:44for the value within this key value
9:45pair, I'll set this to be 1,750,
9:49meaning 1,750 calories. I'm trying to
9:53eat less than 2,000 calories per day. If
9:55you would like, feel free to pick
9:56different numbers.
9:58For day two,
10:01I didn't stick to my diet. On day two, I
10:04had 2,100 calories.
10:07Then for day three,
10:12I had 1,700
10:15calories.
10:17Now to create a series, again, we will
10:19type series equals axis pandas as pd.
10:23call the series constructor, then pass
10:26in our dictionary.
10:30Since our dictionary is made of key
10:32value pairs, we'll use the keys as the
10:35labels.
10:37So, you don't need to pass in an index
10:40like this.
10:44If we were to print our series, here's
10:46the output.
10:49This single column represents the
10:51calories I've eaten each day. And on the
10:54left hand side is each label for this
10:56column. We have day 1, day 2, day 3.
11:01Let's access the lock property to see
11:03how many calories I've eaten on a
11:04certain day. Day 1,750.
11:10Day 2,100.
11:14Day 3 1,700.
11:17Now let's update day three. Perhaps I
11:19cheated on my diet and I had a cookie.
11:22Well, to update a value, we'll access
11:24our series, access the lock property.
11:27We're locating by label
11:30at the label of day three,
11:32we'll add 500 calories,
11:35we'll say plus equals 500. To add 500,
11:40now on day three, the total amount of
11:42calories I've eaten is 2,200.
11:46If I were to print my series,
11:50here's what my chart looks like. 1,750,
11:552,100, 2,200.
11:58Let's delete this line of code.
12:01Now, we'll filter by value. Let's find
12:04any days where I've eaten more than
12:062,000 calories. Which days have I not
12:09followed my diet plan? We'll access our
12:12series. Use the subscript operator.
12:15Return any values within our series
12:17that's greater than 2,00. We'll say
12:20greater than or equal to 2,00.
12:24All right. On day two, I did not stick
12:27to my diet. I had 2,100 calories.
12:31Now, on which days did I stick to my
12:33diet? Return any values within my series
12:37that are under 2,000.
12:41There are two days which I followed my
12:43diet where the calories is under 2,000.
12:46Day 1 and day three. All right
12:49everybody, that's a series. It's a
12:51pandas one-dimensional labeled array.
12:54Think of it like a single column in a
12:56spreadsheet. Now, your homework is to
12:58post a series in the comment section
13:00down below. If you can, try and think of
13:02a unique example. All right, everybody.
13:05And those are series using pandas.
13:14All right, let's do this everybody. We
13:16got to talk about data frames in pandas.
13:18A dataf frame is a tabular data
13:20structure with rows and columns. It's
13:23two-dimensional similar to an Excel
13:25spreadsheet. Here's a beginner friendly
13:28way in which we can create a data frame.
13:30Let's start with a dictionary. We'll say
13:32data equals a dictionary.
13:36For our data, we'll have some employees.
13:38For the key of name, this will be a
13:41column.
13:44Instead of just having a single value,
13:46we'll use a list, a Python list. Here,
13:48we can list a few different names. I'm
13:51going to use Spongebob characters
13:52because most people know who Spongebob
13:54is. We'll have Spongebob,
13:57Patrick,
14:00and Squidward.
14:03Then we'll add another column.
14:07This column will be for the age. How old
14:10is each employee? The value will be
14:12another Python list. I'm going to say
14:15that Spongebob is around 30. Patrick
14:18will be 35. Squidward will be 50. Let me
14:22just do a little bit of organizing.
14:25There. That looks pretty good. Now,
14:28we're going to convert our dictionary
14:30into a data frame. Here's how. We will
14:32assign dataf frame but most people
14:35shorten this to just df meaning dataf
14:38frame. Our dataf frame object equals
14:42access pandas spd. Call the data frame
14:46constructor.
14:48Now the d is capital not lowercase.
14:51That's because this is a constructor
14:53rather than a function.
14:55We're going to pass in our data, our
14:57dictionary.
15:00And then we will print our data frame
15:03which we have named DF for short.
15:09And here's our data frame. It's tabular
15:12data. It's made of rows
15:15and columns.
15:19We're also given an index. This index
15:21has a label for each row starting with
15:23zero. But you can set that to be
15:26something custom.
15:28To do so, when we construct a dataf
15:30frame object, we will set an index by
15:33passing in a keyword argument index
15:36equals a Python list. We'll say employee
15:401, then employee 2,
15:44then employee 3.
15:49And we should have some new labels.
15:51There we go. Employee one is Spongebob,
15:54age30. Employee 2 is Patrick, age35.
15:58Employee 3 is Squidward, age 50. If you
16:01need to select a single row, you can
16:04access the lock property. DF.Lock
16:07subscript operator. Lock means location
16:10by label.
16:12Give me all the data for employee one.
16:16That's the label.
16:19We are given Spongebob age 30. Here's
16:23the label and the data type of this row.
16:25It's an object. Let's select employee 2,
16:30Patrick, age 35. The label is employee
16:332. The data type for this row is an
16:36object.
16:37And employee 3,
16:40we have Squidward, age 50. Employee 3
16:43data type is an object. Now you can also
16:45select by integer location by using the
16:48eyelock property. I lock.
16:52So the index zero will give you the
16:54first row which is again Spongebob. One
16:59is for Patrick. Two is for Squidward.
17:06Let's just print the entire data frame.
17:08Now if you need to add a new column,
17:10here's how.
17:13I'll just add a comment.
17:15Add a new column.
17:20And I'm just going to zoom out a little
17:21bit. To create a new column, we'll
17:24access our data frame. Use the subscript
17:27operator. Then we'll need to think of a
17:29new column name. Let's say job or
17:32position. What does everybody do?
17:36We will assign this to be a Python list.
17:40The number of values must match the
17:42number of rows. We have three rows, one
17:44for Spongebob, Patrick, and Squidward.
17:47In that order, what is the job of
17:50Spongebob? He is a cook. That's the
17:52first value. Let's say Patrick is a new
17:55employee. He doesn't have a position
17:57quite yet. For the time being, we'll say
18:00NA for not available. And then Squidward
18:03is a cashier.
18:06Now, let's print our data frame. And we
18:08should have a new column.
18:10Employee one is a cook. Employee two,
18:13their job is not available yet. And
18:15employee three, employee 3 is a cashier.
18:19That's how to make a new column. Now,
18:21how do we add a new row? Here's how.
18:24Add a new row.
18:28An easy way to create a new row is to
18:30create a new data frame, then
18:32concatenate it. We'll say new
18:36row equals access pandas spd call the
18:41dataf frame constructor again.
18:44Now we're going to pass in a list. For
18:47this list, you can add a dictionary for
18:49each new row you want to add. For
18:51example, if I need to add three rows,
18:52I'll need three dictionaries within this
18:54list. But for now, we'll just add one.
18:58This dictionary needs key value pairs.
19:01For the key of name, we'll add, let's
19:05say, Sandy.
19:09For the key of age, we'll add a value of
19:1228.
19:15Then for the key of job,
19:18we'll add a value of I would say that
19:21SY's some sort of scientist or engineer,
19:23but we'll just say engineer.
19:26This is a separate data frame. It only
19:29contains a single row, a new record for
19:32Sandy. We're going to concatenate it to
19:34our existing data frame. We will
19:36reassign our data frame df equals
19:41access pandas spd. Call the concat
19:44function for concatenate. You may see
19:47people use append, but that's now
19:49depreciated. We'll be using concat for
19:51concatenation.
19:53We will pass in a Python list. Then
19:56include all of the data frames we want
19:58to concatenate. We want to include our
20:01existing data frame and the new row.
20:05And then let's print our data frame. And
20:07we should have a new row. Yes, we have a
20:10new row for Sandy.
20:12However, we have not set an index. The
20:15index was set to zero. Well, we can
20:17manually update that. When we create a
20:20new data frame, you can pass in an index
20:22as a keyword argument. We'll do that for
20:25Sandy.
20:26So, comma, space, and just for
20:29readability, I'll put this on a new
20:31line. Index equals a Python list with a
20:36new label for her of employee 4.
20:42And here we go. Employee 4 is Sandy, age
20:4528. Her job is an engineer.
20:50Now you can add more than one row. Let's
20:53add new rows.
20:56This variable will be new rows. So
20:59within our list, we can add more than
21:02one dictionary. Let's add another row.
21:05I'll do some copying and pasting just to
21:07save a little bit of time. Copy all the
21:10key value pairs and paste it within the
21:12new dictionary.
21:14Let's say the new employee is Mr. Krabs.
21:17His first name is Eugene. Eugene Krabs.
21:21He seems like he's about 60.
21:25His job is that he is a manager.
21:28We'll assign him to be employee 5.
21:35And then we are adding new rows to our
21:38data frame. All right, let's take a
21:40look.
21:43There they are. We have two new rows.
21:45one for Sandy and another for Eugene.
21:48Sandy is 28. She's an engineer. Eugene
21:51is 60. He's a manager. All right,
21:54everybody. Those are data frames. Dataf
21:56frames are a tabular data structure.
21:59They contain rows and columns. Your
22:02homework is in the comment section to
22:04post the output of a dataf frame. Try
22:07and think of a unique example if you
22:08can. And well everybody, those are dataf
22:11frames using pandas.
22:19Okay, let's do this everybody. So, in
22:21this video I'm going to show you how we
22:23can import CSV and JSON files using
22:26pandas. You will need some data to work
22:28with. For this demonstration on my
22:31desktop, I have two files. One is a CSV
22:34file, meaning commaepparated values, and
22:37the other is a JSON file.json.
22:41For this demonstration, you'll need some
22:43data to work with. I looked online and I
22:46did find a CSV file that contains the
22:48stats for the original 150 Pokémon. For
22:51this video series, I'll be using Pokémon
22:53because many of us have grown up with
22:55Pokémon. We can relate to it, but feel
22:57free to use any kind of data that you
22:59would like to work with. Just make sure
23:01that it's in a CSV format.
23:06So for my data for number one we have
23:09Bulbasaur all the way to number 150
23:12where we have Mewtwo.
23:14But again the data you want to use is up
23:16to you. If you would like a copy of this
23:18data I'll post it in the comment section
23:20on YouTube. You can just copy and paste
23:22it like so. Then to create a CSV file
23:27wherever you would like to place this
23:28file. Right click, go to new text
23:31document.
23:33Open it. Paste your CSV data.
23:37Save it. Then be sure you save it as a
23:39CSV file. For example, data.csv.
23:45Then once you have your CSV file for
23:48convenience, I'll move it to my Python
23:49project folder.
23:52My CSV file is right next to my main
23:54Python file. We're going to create a
23:57data frame using our CSV data. DF
24:00meaning dataf frame equals access pandas
24:03as pd
24:05call the read
24:08csv function and then we have to pass in
24:11either a relative or absolute file path.
24:14This file is right next to my main
24:16Python file. I just need the file name
24:18including the extension. In this case,
24:21it's data.
24:23CSV.
24:24Let's take a look at our data frame. I
24:26will print my data frame. And here's
24:29what we have.
24:32I have 150 rows and seven columns.
24:37The data we're outputting, it's
24:39truncated. If you print the data frame
24:41directly, you'll be given the first five
24:43rows and the last five rows. Here I have
24:48the Pokemon from Bulbasaur to
24:50Charmeleon, numbers 1 through 5, and the
24:53last five, Moltres to Mewtwo, numbers
24:56146 and 150. If you need to print all
25:00the data in your dataf frame, to print
25:03the entire data frame, all the rows and
25:05columns, and not a truncated version,
25:07following our data frame, we will call
25:09the two string function. Be careful when
25:13working with a very large file. You'll
25:15print everything.
25:17This file isn't that big. We only have
25:20150 rows. So now we display all the data
25:22for our Pokémon.
25:25A few of the columns include their
25:27number in the Pokédex, their name, their
25:30first type such as grass, fire, water,
25:33their second type if they have one.
25:35Bulbasaur, Ivysaur, Venusaur are also a
25:37poison type, but many of them are
25:39missing a second type such as
25:41Charmander. Charmander doesn't have a
25:43second type, so the value here is not a
25:45number. nan. There's a height in meters,
25:49a weight in kilograms, and a legendary
25:52status, which is either zero or one. If
25:55this Pokémon is not legendary, they have
25:57a legendary value of zero. If they are
25:59legendary, for example, Mewtwo,
26:03Mewtwo has a legendary status of one,
26:06meaning that there's only ever one
26:07Mewtwo.
26:09That's what it means by legendary.
26:12Now to read a JSON file. Here's a JSON
26:15file that contains the same data. JSON
26:18means JavaScript object notation. For
26:20entry number one, we have Bulbasaur,
26:23Ivysaur, Venusaur all the way till
26:25Mewtwo, which has a number of 150. For
26:29convenience, I'll place this JSON file
26:31within my project folder.
26:34And then to read a JSON file.
26:37To read a JSON file, we will call the
26:40read JSON function.
26:43And then pass in the name of that file,
26:45including the extension
26:47data.json.
26:51And here's what we have. We have our
26:53same data again. All the original 150
26:56Pokémon. All right, that's how we can
26:58read a file using pandas. I couldn't
27:01think of any homework for you to do. So,
27:03to increase engagement with this video,
27:05post the name of your favorite Pokémon
27:07down in the comment section. All right,
27:09everybody. And that is how to read both
27:10CSV and JSON files using pandas.
27:19Why? Hello again everybody. In this
27:21video, I'm going to show you different
27:22selection techniques using pandas. In
27:25the previous lesson, we've loaded a CSV
27:27file. Here's what my data frame looks
27:29like. My data frame contains the
27:31original 150 Pokémon all the way from
27:34Bulbasaur to Mewtwo. All right, so we
27:37have a data frame. To make a selection
27:39by column, I'll just add this as a
27:42comment. Selection by column.
27:46What we'll do is print access our data
27:49frame. Use the subscript operator. Then
27:52we will pass in a column name. We are
27:55going to pass in the name column. And
27:59here's what we have. As a single column,
28:02we have all the names. And this is
28:04actually a series. If you have a lot of
28:06data to print, the default behavior for
28:09pandas is to print the first five rows
28:12and the last five rows. What you see
28:14here is a truncated version. If you need
28:17to print the entire thing, follow this
28:19with the two string function. Do be
28:23careful if you have a lot of data you're
28:25working with.
28:27And here are the names of the original
28:29150 Pokémon.
28:32Let's say we need the height, the height
28:34column.
28:37Well, I'm going to do a search for
28:39height. I will place that within the
28:41subscript operator.
28:44And here's the height of every Pokemon
28:46in meters
28:48or the weight.
28:53We are searching for the column of
28:55weight.
28:58And here are all the weights.
29:01This one's heavy. Whatever it is, I
29:02think that's Venusaur. 100 kg. Now to
29:06select multiple columns, we are going to
29:09print access our data frame. Then within
29:11the subscript operator, we'll pass in a
29:14Python list. A list of all the columns
29:17we would like to output. Let's output
29:19the name column,
29:21the height column,
29:24and the weight.
29:33We have three columns, the name, the
29:36height, and the weight. Then to display
29:38all the data, all the rows, you can
29:41follow this with the two-string
29:43function. Two string
29:50And we get all the rows and these three
29:52specific columns name, height, weight.
29:56All right, that is selection by column.
30:02Now we'll make a selection by row or
30:06rows.
30:07Let's take a look at our data frame.
30:11I'm going to print my data frame.
30:14Each of these rows has a label. The
30:17default behavior for pandas is to give
30:19each row a number for the label starting
30:21with zero. They're in ascending order.
30:24Well, we can access the lock property,
30:27then pass in a label.
30:30So since I didn't set a label for each
30:31of these rows, zero would correspond
30:34with the first row where we have
30:35Bulbasaur
30:39one would be Ivysaur.
30:42But optionally what you could do,
30:45I can set one of the columns to serve as
30:48the index so that you can access one of
30:51the names by label using the lock
30:52property. You're probably not going to
30:54remember which Pokémon is 145, but
30:57you'll remember it by its name, so it's
31:00easier to look up. So, what you could do
31:02is that when we read our CSV file, we
31:05can pass in a second argument, a keyword
31:08argument, index
31:11meaning column. Index column equals
31:15name. Then, let's take a look at our
31:17data frame.
31:21Now, we're setting the label of each row
31:23to be the Pokémon's name. We can do a
31:26search by name pretty easily. So, to set
31:28that up, you just pass in a keyword
31:30argument of index column, index call for
31:33short, then set which column you would
31:35like to serve as the index.
31:38So, then when we access our data frame,
31:40we will access the lock property,
31:42meaning location by label. Then, let's
31:45do a search for Pikachu. We'll pass in a
31:47string of Pikachu.
31:50And here's all the data for Pikachu.
31:52Pikachu is number 25. Type one is
31:55electric. Pikachu doesn't have a second
31:58type. That's nan, not a number. The
32:00height of Pikachu is 0.4 m. Weight 6 kg.
32:04Pikachu is not a legendary Pokémon. Here
32:07you have the name and the data type of
32:09what Pikachu is. It's an object.
32:12All right, let's do a search for
32:15Charizard. Charizard.
32:19And here's Charizard. Charizard is
32:21number six. Type one is fire. Type two
32:23is flying. Height 1.7 m. Weight 90.5 kg.
32:28Not legendary. If you don't want all the
32:30data when displaying your row. When
32:33locating by label, we can pass in a
32:36second argument. We can pass in a Python
32:39list of all the columns we would like to
32:41select. So with Charizard, I would just
32:43like the height
32:48and the weight.
32:51So we will pass in a Python list as the
32:54second argument that contains the
32:55columns we would like to display.
32:58There we go. With Charizard, we only
33:01have the height and the weight, those
33:03two columns.
33:04Now, you also can select a range of rows
33:07as well. After the Pokemon's name,
33:09you're going to use the slice operator,
33:11which is a colon. Give me everything
33:14between Charizard and
33:17let's say Blastoise.
33:20Blastoise.
33:23There we go. We have Charizard,
33:25Squirtle, Warro, Blastoise.
33:29You can also use integer-based indexing
33:32if you so choose.
33:34We're going to print access our data
33:36frame access by integer location.
33:42Let's say you need the first 10. You
33:44would pass in those indices 0 colon 11.
33:49Remember that the second number is
33:50exclusive. You would then be given the
33:53first 10 rows
33:55between Bulbasaur and Metapod.
33:59Maybe you need every second row. Then
34:02add a step of two.
34:05Now we're given every second row. You
34:07can see that the number is going up by
34:09two. As a second argument, you can
34:12select the indices of which columns you
34:14would like. I would like the first three
34:17columns. 0 col 3.
34:21With the range of 10 indices that I've
34:23selected, I specified I would only like
34:25the first three columns. Besides our
34:28index with all of our unique labels, we
34:30have the first three columns, each
34:32Pokemon's number in the Pokédex, their
34:34first type, and their second type if
34:36they have one.
34:39All right, now we'll go over an
34:40exercise.
34:42We'll have a user type in the name of a
34:44Pokémon, and then do a search for it.
34:46We'll create a variable of Pokemon
34:49equals call the input function to ask
34:52for some user input. Enter a Pokemon
34:56name.
35:00Using this technique, if the Pokemon's
35:02name doesn't exist, it's going to cause
35:04a key error.
35:06Using a try block, we will try and print
35:11access our data frame. Access the
35:14location property.
35:16Then instead of placing a string here
35:18with a Pokemon's name, such as Pikachu,
35:20we'll place in the string variable of
35:22Pokemon,
35:24we'll do a search for whatever the user
35:26types in. Now, if that Pokemon doesn't
35:29exist,
35:31we will catch an exception if it
35:32happens.
35:34Except key error,
35:39where we will print, I'll use an F
35:41string. We'll say that Pokemon's name.
35:46Pokemon not found. And now we can do a
35:50search for a Pokemon. Let's test it.
35:53Enter a Pokemon name. I'll type in
35:55Pikachu. Make sure that the first letter
35:57is capital. And here's the stats for
35:59Pikachu.
36:01Let's do a search for Snorlax.
36:08And here's the stats for Snorlax.
36:11What if there's a Pokemon that doesn't
36:12exist, like whatever that is? Well, this
36:17Pokemon was not found.
36:20I couldn't think of any homework for you
36:21to do. So, let me know what your
36:23favorite non-coding hobby is. Post it
36:26down in the comment section. And that is
36:28selection using pandas.
36:36Hey, what's going on everybody? I'm
36:38going to explain filtering using pandas.
36:40If I could summarize what filtering is,
36:43it's basically keeping the rows that
36:45match a condition when working with the
36:47data frame. In the previous lessons,
36:50we've been working with the data frame
36:52that contains the original 150 Pokémon.
36:55Make sure that you have some data to
36:56work with before starting this lesson.
36:58Let's say I would like to create a new
37:00data frame and filter any Pokémon that
37:03have a height above 2 m. we would have
37:05to access this height column and return
37:08any rows where the height is 2 m or
37:10above. Here's how we can do that. Let's
37:14create a new data frame. We'll call it
37:16tall Pokémon equals. We will access our
37:20data frame. Use the subscript operator.
37:23Let me zoom in a little bit. We're going
37:26to return a series. Access our data
37:28frame. Use the subscript operator. We
37:32will select the column of height.
37:36Then we will write a condition. Return
37:39any rows where the height is greater
37:41than or equal to two.
37:46And then we will output our new data
37:48frame tall Pokémon. And here's what we
37:50have.
37:52We have filtered out any Pokémon that
37:55have less than 2 m for height. You can
37:57see that with the height column, all the
37:59heights are 2 m or above.
38:02So really with filtering, we're just
38:04keeping the rows that match condition.
38:06Let's apply another filter. Let's say
38:09heavy Pokemon.
38:14Heavy Pokémon equals access our data
38:17frame. Use the subscript operator. We
38:20will select the column of weight. Access
38:22our data frame. Subscript operator. We
38:26will select the column of weight.
38:30return any Pokemon that have a weight
38:32that's greater than let's say greater
38:34than 100
38:37and then we will print our heavy Pokémon
38:40that new data frame
38:42and here we go. We've returned all the
38:45rows where the weight is over 100.
38:51Now let's return any Pokémon that are
38:53legendary. If a Pokémon's not legendary,
38:56this value is zero. If it is legendary,
38:58it's one.
39:03We'll create a new data frame of
39:04legendary
39:07Pokemon
39:08equals access our data frame. Use the
39:11subscript operator. We will select the
39:14column of legendary.
39:19Now I set this equal to be one if it is
39:21legendary.
39:25We have four legendary Pokemon.
39:27Articuno, Zapdos, Moltrace, Mewtwo.
39:31Now, what you could do for readability,
39:33the boolean value of true also equals 1.
39:38That would work too. If your CSV data
39:41contains booleans, mine doesn't, you
39:44don't necessarily need double equals
39:46true. That's redundant. Since my
39:48original CSV data contains zeros and
39:50ones, I would need that double equals
39:55Let's find any water Pokemon.
39:58Water Pokemon equals access our data
40:02frame subscript operator.
40:05We will select the column of type one.
40:09If type one is equal a string of water,
40:15then we will return any water Pokémon.
40:17Now, this is only for type one.
40:20We have Squirtle all the way down to
40:23Vaporeon.
40:25But what if their second type is water
40:28and not their first type? Well, we could
40:30use the or logical operator.
40:32If you're going to use a logical
40:34operator, be sure to enclose your first
40:36condition within a set of parenthesis.
40:39With the or logical operator, this is a
40:42C style operator.
40:44With Python, it would be or.
40:47But we're going to be using the C style
40:49operator which is a vertical bar for
40:51ore.
40:53We'll write another condition. Access
40:55our data frame. Select the column of
40:58type two
41:00where the second type
41:02is equal to water.
41:06If the first type is water or the second
41:08type is water, return that row.
41:14See, we have more rows. There's a few
41:16Pokémon that have a first type that's
41:18not water, but their second type is
41:20water.
41:22Now, we'll use the and logical operator.
41:26Let's find any Pokémon that are fire and
41:29flying. We'll name this new data frame
41:31just FF for short. Fireflying.
41:34Fireflying Pokémon equals access our
41:38data frame. We'll do a search for any
41:40fire Pokémon. Select the column
41:44of type one.
41:46Check to see if that's equal to a string
41:49of fire. Let's see what we have
41:52currently.
41:54Here's all the fire Pokémon. Starting
41:56with Charmander all the way to Moltrace.
42:01Now, we'll see if their second type is
42:03flying. We'll be using a C style and
42:05logical operator, which is an amperand.
42:09Be sure you're not using the Python and.
42:12If we're using a logical operator,
42:14enclose your conditions within a set of
42:16parenthesis.
42:18Now, for my second condition, access our
42:21dataf frame subscript operator. Select
42:24the column of type two.
42:27Let's check to see if type two is equal
42:29to flying.
42:33Now, we're returning any fire and flying
42:35Pokémon. And there's two, Charizard and
42:39Moltrace. Their type one is fire. Their
42:42type two is flying.
42:45All right, everybody. That is filtering.
42:47Basically, you're keeping the rows that
42:49match a condition. For your homework, to
42:52help increase engagement with this
42:53video. In the comment section, post a
42:55data frame where you search for two
42:57different types. Try and pick some
42:59different types that we haven't used
43:00yet. And well, everybody, that is
43:02filtering using pandas.
43:10In this video, I'm going to explain
43:12aggregate functions using pandas.
43:14Aggregate functions. They reduce a set
43:17of values into a single summary value.
43:20Basically, we're reducing a bunch of
43:22values into one. We use aggregate
43:25functions to summarize and analyze data.
43:27In this demonstration, I have a data
43:29frame that contains the original 150
43:32Pokémon. Let's say for some of these
43:34columns such as the number, the height,
43:37the weight, and legendary status, I
43:39would like to find the mean or the
43:41average. To find the mean for all
43:43numeric columns, we can do the
43:45following.
43:46When we output our data frame, we will
43:48call the mean aggregate function. But
43:51there's one thing we're missing.
43:54We can't use the mean function on any
43:56columns that are non-numeric.
43:59We can't find the mean for anything
44:00that's non-numeric such as Pokémon
44:03names. They're alphabetical characters.
44:06What we could do is that when we call
44:08the mean function, we can pass in a
44:10keyword argument
44:12numeric_on
44:16equals true.
44:18We're telling pandas find the mean of
44:20any columns that are numeric. And here's
44:23what we have. The average number is
44:2675.5. The average height is 1.2 m. The
44:30average weight is 46.2 kg. The average
44:34amount of legendaries is 02 2%.
44:39So we have mean. Let's do some copying
44:42and pasting. Another aggregate function
44:46is sum. Let's sum all the numeric
44:49columns.
44:50And here's the sum. the sum of all the
44:52numbers, the sum of all the heights, the
44:55sum of all the weights, and the sum of
44:57the numbers of all the legendaries.
45:00Let's find the minimum.
45:03We will use the min aggregate function.
45:07The minimum number is 1. The minimum
45:10height is 2 m. The minimum weight is.1
45:13kg. And the minimum legendary status is
45:16zero. Then we have max.
45:21We will call the max aggregate function.
45:25Here are the max values. 150 8.8 460
45:301.
45:33We also have count.
45:38With the count function, we don't need
45:39to pass in a keyword argument. We are
45:42counting the number of values within
45:44each column. 150 150 150. Now, not every
45:48Pokémon has a type two. That's why this
45:50is only 67. Count won't include any null
45:53values. That's why this number is 67 and
45:56the rest is 150.
46:00That's using aggregate functions with an
46:02entire data frame. These aggregate
46:04functions would apply to the whole data
46:07frame. The next set of aggregate
46:10functions will be for a single column.
46:14All right, just to save a little bit of
46:15time, let's do some copying and pasting.
46:18We'll copy our five previous print
46:19statements and paste them.
46:22Now rather than selecting the entire
46:24data frame, we will select a single
46:26column by using the subscript operator.
46:30Then selecting a column, let's pick
46:32height. What is the average height?
46:35Since height is a numeric column, we
46:37don't have to pass in this keyword
46:38argument. What is the mean of the height
46:42column?
46:43That is 1.2. 1.2 m.
46:47Then let's do this with sum
46:50subscript operator. Select the column of
46:53height and we don't need to pass in that
46:55keyword argument. What is the sum of all
46:58the heights? 180 m.
47:02Then let's do this with min. What is the
47:04minimum value? What is the minimum
47:06height in this column?
47:11That is 0.2 m.
47:15subscript operator we will select the
47:17column of height and we don't need to
47:19pass in this keyword argument with our
47:22column of height the maximum value is
47:248.8 8 m.
47:27And what's the count?
47:29Select our column of height
47:33and the count
47:35is 150. That's 150 non-null values.
47:41But if I was to pick a different column
47:42like type two, well, if we remember that
47:45was only 67.
47:48All right, that's how to use aggregate
47:49functions with a single column. Select
47:52your data frame. Then select the column
47:54using the subscript operator.
47:56Now there's also something called group
47:59by. Group by is a function.
48:02Now if I was to print my data frame, I
48:05will print my data frame.
48:08We can arrange our data frame into
48:10different groups. We can group together
48:12rows by something that they have in
48:13common. Let's group our data frame by
48:16each creature's type, their type one.
48:18We'll put all the grass types in one
48:20group, all the fire types in another
48:23group, and we'll do this with all the
48:24different types.
48:28We will do this by creating a group by
48:31object.
48:32Group equals access our data frame. Call
48:36the group by function. Then we're going
48:40to pass in the name of a column by which
48:42we want to group these records by.
48:46We'll group by type one.
48:48Now, if you were to output your group
48:50directly,
48:52here's what you would see.
48:56Group is an object. When you apply an
48:59aggregate function, the group object
49:01will return a series for you to work
49:03with and display. Let's take our group,
49:06select the height column,
49:09then call an aggregate function. What is
49:11the average height for each group?
49:16and we are grouping by type one.
49:19Here's what we have. Here are the
49:20average heights for bug types. The
49:23average height is 0.9 m. Dragon types
49:26have an average height of 2.6.
49:29So, we are grouping by each Pokémon's
49:31type. Their type one.
49:36Let's find the sum.
49:38We are grouping by type one. Let's find
49:40the sum.
49:44All of the bug types have a combined
49:46height of 10.8 m. All the dragon types
49:49have a combined height of 8 m.
49:52Seems like water types have the greatest
49:54combined height.
49:58Let's find the minimum value from each
50:00type.
50:02We will call the min function.
50:05And here's the minimum values for each
50:07type or the max.
50:11Here
50:17are all the max values for each type
50:22and the count.
50:30Here's the total number of bug types,
50:3212. Dragon types, three. Electric types,
50:359. So on and so forth. There's a lot of
50:38water types I noticed.
50:41All right, everybody. So, those are
50:43aggregate functions. They reduce a set
50:46of values into a single summary value.
50:49They're used to summarize and analyze
50:51data. And well, everybody, those are
50:53aggregate functions using pandas.
51:00Guess what? It's me again. So, today I'm
51:02going to explain data cleaning using
51:04pandas. Data cleaning is the process of
51:07fixing or removing incomplete, incorrect
51:10or irrelevant data. Approximately 75% of
51:14the work that is done with pandas is
51:16specifically for data cleaning. You do a
51:18lot of it. In this demonstration, I have
51:20a data frame that contains the original
51:23150 Pokémon. Now, let's say you would
51:25like to drop a column because it's not
51:27relevant. We're going to take our data
51:30frame, call the drop function as a
51:33keyword argument. We're going to pass in
51:35columns equals. Then within a Python
51:38list, you can list all the columns you
51:40would like to drop. I would like to drop
51:42that column of legendary.
51:45This function of drop does return a new
51:47data frame. So we're going to reassign
51:49our current data frame. Dataf frame
51:52equals then call the drop function of
51:55the previous data frame. And let's take
51:57a look. We should be missing that
51:59column, which we are. Now, let's say you
52:02don't want the number.
52:04Well, we can add that to the Python
52:05list. Drop. I think it was N O.
52:11Yes, that column is also gone. That's
52:15how to drop irrelevant columns.
52:18Here's how to handle missing data in my
52:21data frame. There are many Pokemon that
52:24don't have a second type, but all
52:26Pokemon do have a first type. For
52:28example, Charmander. Charmander is a
52:30fire type, but Charmander doesn't have a
52:33second type. It's a pure fire type. If a
52:37row is missing a value within type two,
52:39we're going to drop that entire row.
52:41Here's how to do that. We're going to
52:44reassign our data frame,
52:46access our data frame, call the drop na
52:50method.
52:53We're going to pass in a keyword
52:54argument of subset equals a Python list.
52:59Any columns we place within this list
53:01will drop those rows if they're missing
53:03a value.
53:05Within type two, drop any rows that are
53:08missing a value.
53:11When I display my data frame, I'm going
53:13to call the two string function just so
53:14we can see the whole thing.
53:17Two string function.
53:22There should be no rows for Pokemon that
53:24are missing type two. You can see that
53:26they're numbered here. One, two, three.
53:28We go to six because four through five
53:31are missing. And then we jump up to 12
53:33where we have Butterfree. We have
53:35removed any rows where there are null
53:37values within type two.
53:41In place of dropping any rows where the
53:44values are not available, let's replace
53:46them instead with a different function.
53:48We will reassign our data frame. dataf
53:51frame equals access our dataf frame call
53:54the fill na function fill any not
53:59available values within this function
54:02we're going to pass in a dictionary
54:04within this dictionary we'll select a
54:06column we'll select type two again
54:11replace any not available values within
54:14the column of type two with the
54:16following value instead of not a number
54:19nan and we'll replace those values with
54:22none.
54:24And let's take a look
54:30for type two for any Pokémon that don't
54:32have a second type. We've replaced any
54:35missing values such as not a number with
54:37the word none.
54:40That's how to replace any missing
54:42values.
54:44Number three, we're going to fix any
54:46inconsistent values.
54:49We can replace any instance of a value
54:52with another one within type one. Let's
54:55select any grass types. We'll change any
54:57instance of grass to be all uppercase
54:59letters. Here's how you can do that.
55:02We're going to take our data frame. Then
55:04select a column. Within the column of
55:07type one,
55:09this will be a series that we're
55:10selecting. We're going to access that
55:14column using the subscript operator. We
55:16are selecting the column of type one.
55:20Call the replace function and then we
55:23will pass in a dictionary.
55:26Let's replace any instance of grass with
55:30the following value. This is a
55:32dictionary. We need a colon then space.
55:35We'll replace grass with all capital
55:38grass to test it. And that should
55:41replace any instance of grass within
55:43type one.
55:45There we are.
55:47Let's find a few more.
55:51There's more grass types. Oddish, gloom,
55:53vile plume.
55:56This won't change any instance of grass
55:58for type two because we've only selected
56:00the type one column.
56:02Let's change a few more. Within our
56:04dictionary, we can pass in more than one
56:06key value pair. Let's replace any
56:08instance of fire with fire all caps.
56:15Let's do the same with water.
56:17Replace any instance of water within
56:19type one with water all uppercase.
56:25Yeah, it looks good. Grass is all
56:27uppercase. Fire is all uppercase.
56:31Water is all uppercase.
56:34No other types are all uppercase.
56:37They're capitalized normally.
56:41Moving on, we can standardize text. Just
56:45to demonstrate, within our name column,
56:48let's take all the individual names and
56:50make them lowercase. Here's how you can
56:52do that.
56:54We will select the column of name. Dataf
56:57frame subscript operator. We are
56:58selecting the column of name. This is a
57:01series. Equals. Select that column
57:04again. Dataf frame subscript operator.
57:07We are selecting name. that series
57:11access string call the lower function
57:14and that's going to take our column of
57:16name and make all of the characters
57:19lowercase.
57:21So that's how you can standardize some
57:23text. Number five, I'm going to show you
57:26how we can fix or change data types with
57:29my column of legendary.
57:31Non-leendary Pokémon have a value of
57:33zero. Legendary Pokémon have a value of
57:36one. I think booleans would make more
57:39sense to use. We'll change these values
57:41to be booleans. Here's how. We'll select
57:44a column, access our data frame,
57:47subscript operator, select the column of
57:50legendary. This is a series. We're going
57:53to reassign it. Access that column
57:55again. Dataf frame subscript operator
57:58legendary. And I'm just going to copy
58:00this because I'm lazy. We'll call the as
58:04type function. and then pass in a new
58:06data type. For booleans, you type bool.
58:10And now for the legendary status, it
58:12should all be booleans.
58:14Dratini, dragon, dragonite, they're not
58:17legendary Pokémon. That's false. But
58:19Mewtwo is a legendary Pokémon. That's
58:22true. Same thing applies to Articuno,
58:25Zapdos, Moltrace.
58:29Here's how to remove duplicate values.
58:32Going to our CSV data. Let's add a few
58:35extra copies of Pokemon. Let's add a few
58:37more copies of Bulbasaur.
58:40I don't know, a total of four is good.
58:43And then a couple for Mewtwo.
58:48We have duplicate entries.
58:52Let's print our data frame just to be
58:54sure that we do have the duplicates.
58:57Yeah, we got four entries for Bulbasaur.
59:00And then at the bottom we have three
59:03entries from U2.
59:06To remove any duplicate values we will
59:08reassign our data frame. Dataf frame
59:10equals access our data frame. Call the
59:13drop duplicates function.
59:17Duplicates.
59:19And that's all you got to do. And now we
59:22should be missing those extra entries.
59:24Yeah, there's only one Bulbasaur
59:26and there's only one Mewtwo.
59:29All right everybody. So that is data
59:31cleaning. It's the process of fixing or
59:33removing incomplete, incorrect or
59:36irrelevant data. On average, 75% of the
59:40work done in pandas is data cleaning.
59:42It's an important topic. I couldn't
59:44really think of any homework for you to
59:46do. So to increase engagement with this
59:48video, I don't know, post your favorite
59:50anime in the comment section. And well
59:52everybody, that is data cleaning using
59:54pandas.

About this transcript

This page contains the full transcript of Learn Pandas in 1 hour! 🐼 by Bro Code, generated from the public captions YouTube serves with the video. The transcript has 8,014 words across 1,289 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.