YT YouTube2Text

YouTube2Text → Transcripts → Deduplication (DDB) – Interview-Focused #commvault #interview #cloudcomputing #backup #technology

Deduplication (DDB) – Interview-Focused #commvault #interview #cloudcomputing #backup #technology — Transcript

by TechVault by abhi · 2,558 words · 499 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00Hi everyone, welcome back to our YouTube
0:02channel. Today we are going to discuss
0:04uh some more questions of Commvault
0:07interview. If you're going for the
0:09interview, especially for the Commvault
0:10and backup, then uh the interview will
0:12definitely will
0:14ask a question from this part, that is
0:16uh DDB, okay?
0:18Deduplication database. What is DDB?
0:21Deduplication
0:24database.
0:27Okay?
0:27So uh interview will generally ask a
0:30multiple questions from this section,
0:32that is uh
0:33I mean, what is the role of
0:34deduplication database in Commvault?
0:37Explain the difference between client
0:39side and media agent side deduplication.
0:42How does Commvault identify the
0:43duplicate data blocks or the chunks?
0:46Okay? How does DDB pruning work? What
0:48are the best practices if the DDB layout
0:51or the performance optimization? I mean,
0:53if you're facing some uh performance
0:55issues, okay? And uh
0:58how do you monitor DDB health and growth
1:00over the time?
1:02And the most important questions, if you
1:04face any uh challenges, if you face any
1:07issues on DDB, then what logs or the
1:09tools do you use to troubleshoot DDB
1:12performance issues? So, there is a
1:14multiple questions interview will be
1:15asked in your interview. So, we must
1:18know the what is a DDB and how it works
1:20in Commvault, okay? First, we are uh
1:25Okay.
1:26Deduplication database. So, it is stand
1:29for the deduplication
1:32database.
1:34If you break this point, that is the
1:38duplicate.
1:41Okay? That means
1:43it
1:44remove the duplicate data, remove the
1:47duplicate chunks, remove the duplicate
1:49blocks
1:50during the backup operation, okay? So,
1:53if anyone ask what is the deduplication
1:55in database, I mean, what is the
1:56duplicate What is the D D B in
1:58Commvault? Then
2:00you can you can tell them deduplication
2:03is the process of identifying and
2:06removing duplicate chunks or blocks of
2:09data. Kiska? Of data during the backup
2:13operation.
2:14That means during the backup operation,
2:16what D D B will do, D D B identify and
2:19remove the duplicate chunks or blocks of
2:22data during the backup operation.
2:24Okay? So,
2:26why we need it?
2:28Then
2:29interview will ask, why we need to
2:31enable the D D B?
2:33Then you tell them for the storage
2:36optimization.
2:38Storage optimization. Let's an example.
2:42There's a
2:43lemon language. There's a two storage
2:45policy.
2:47I will cover this part as well. There's
2:49a two storage policy. There's a S1
2:51storage policy and there's a S2 storage
2:53policy.
2:55Okay?
2:57And here we enable the deduplication.
3:01Here we not enable the D D B.
3:04Hm?
3:05And
3:06uh there's a environment where we are
3:08taking the backup of
3:10um where we taking the backup of 100
3:13virtual machines, 100 VMs.
3:17And in 100 VMs, each VMs have the size
3:20of 50
3:22GB.
3:23So, that means you need
3:25you need 5 TB.
3:29Okay? You need 5 TB.
3:33What happen when this
3:36is attached to S1?
3:39Let's an example, it's
3:40saving the 80% and somehow 70%.
3:45Okay?
3:46So, there's a some I mean, if you are
3:48going to end up create a D D B and
3:50there's some percentage all seen your
3:52lab. Okay? So, so let's an example, it's
3:56saving an 80%
3:59after deduplication. I mean, after
4:00removing the old chunks, old data
4:03from the
4:05from from the I mean, at the time of
4:08backup operation,
4:10then you need only 1 TB.
4:13And here we need 5 TB.
4:16There is a lot of difference.
4:18Okay? That's why we need to
4:21DDB enabled.
4:23Okay?
4:24So,
4:25if it's not, I mean, at the time of DDB,
4:28they only retain the
4:30unique value,
4:32okay? Then, what about the modified
4:33value? They
4:35the this this is the next part.
4:37Deduplication database store the
4:38metadata about these unique data chunks
4:42and allowing the Commvault to reference
4:44previously stored block instead of
4:47saving them again and again.
4:50Okay?
4:51So, this dramatically
4:53reduce the storage consumption and
4:55improve the backup efficiency as well
4:56because there is a multiple factors in
4:58which the in which the backups are
5:01dependent, okay?
5:04What are the components we are
5:06we are going to use in
5:08DDB? So, there is a multiple component,
5:11but
5:12uh how they are managed, how DDB are
5:14managed. So, there is a multiple things,
5:16so, okay? So, if you're talking about
5:18the uh
5:20how did we deduplication are managed?
5:22So, it is generally managed by the
5:23storage policy.
5:26In the storage policy, we have the data
5:28paths.
5:30We have data paths.
5:32We define the retention period.
5:35The okay, there is a one question. What
5:37is retention period?
5:39How What is retention period? So,
5:42retention period is nothing but how data
5:44long you kept in your environment, your
5:47storage.
5:48Okay? Before it deleted automatically. I
5:51mean, before it deleted
5:53uh deleted automatically. Okay? Before
5:55their complete retention period.
5:56Something.
5:59Okay, let's jump to the duplication
6:00part.
6:01So,
6:02uh how do There is one question for you
6:06when the interviewer ask
6:08how duplication manage through is
6:09through duplication centrally manage
6:12through the storage policy.
6:14There's a multiple component SIDB store.
6:17You can Google it as well. SIDB engine.
6:23Okay.
6:24DDB.
6:26If you known as the SIDB database,
6:29DDB partition. There's a multiple.
6:32Why we use partition? I will explain in
6:34coming lecture.
6:36Okay.
6:38Duplication centrally manage through the
6:39storage policy.
6:41And each storage policy can maintain own
6:44deduplication setting can associate with
6:46the global deduplication.
6:49Okay, there is a two
6:50uh independent and the global. Global
6:52deduplication storage policy depending
6:54upon the type of the data and the
6:55production size. This depends upon
6:57generally depends upon the product size.
6:59Production size.
7:01Okay.
7:02If you want to uh use the dedicated
7:03storage policy for the duplication, it
7:05create automatically. Let me Let me
7:07explain. A dedicated A dedicated
7:10deduplication storage policy consists of
7:12one library.
7:13This is the main part. One deduplication
7:15database, one or more media agents.
7:19For scalability and purpose using a
7:20dedicated deduplication policies allows
7:23for the
7:24efficient movement of very large amount
7:25of data. Actually, depends upon the uh
7:28environment. In which environment you
7:29have worked or in
7:31uh or which environment you want to
7:32create a DDP.
7:33Okay. When backing up the large amount
7:36of data
7:37with separate data types that do not
7:39duplicate
7:41well against each other. I mean, it's
7:44a war between
7:46within a family, you can say that. Such
7:48as database are the file system.
7:51If you are enable the horizontal scaling
7:53for applications, it
7:56This is the important part. The
7:58dedicated policies are created
7:59automatically. You do not create you
8:02know you did not I mean you did not want
8:04to create anything and any click on the
8:07console level.
8:08Okay?
8:10Global deduplication,
8:12if you
8:13recall them global
8:16then it is that means is will imply it
8:18will apply on the entire.
8:21Okay? So, global deduplication is
8:23storage policy provides a one large
8:25global deduplication database which can
8:27be shared by the multiple deduplication
8:29storage policies.
8:32Keep in mind that deduplication managed
8:34by the storage policy always.
8:36Okay?
8:38Each storage policy can manage the
8:39specific content with with own retention
8:42rules. Why
8:44I explained the retention?
8:46Here you can correlate this one. Okay?
8:49However, all participating storage
8:51policies copies share the same
8:53uh data paths.
8:56Which consists of media agent, disk
8:57library, and the global deduplication
8:59database. I hope you
9:02uh you are able to correlate everything.
9:05Okay? If interview ask you interview
9:08that can you explain the
9:10key features and key difference between
9:13the DDB policies and global DDB
9:16policies, then you can explain
9:19two or three points here. Apply to the
9:22specific storage policy or the media
9:23agent if you talk about the DDB
9:25policies.
9:26And if you talk about the global DDB
9:27policies, then it's shared across the
9:29multiple storage policy and the media
9:31agent. There's a key difference.
9:33Request individual configuration per
9:34policy. Centralized management for the
9:37duplication is through across the
9:38policies. Ideal for isolated isolated
9:41workloads are dedicated backup targets.
9:44It is isolated is a it will work for the
9:46larger environment.
9:48Okay. Best for the large environment
9:50needing the unified applications. This
9:52is spelling mistake.
9:54Okay.
9:56Got it?
9:58So, how it works?
10:01I mean,
10:03if backup I mean, if for the backup
10:06operation started, then what is the
10:08workflow of DDP?
10:10Okay. Client backup the data.
10:13Data injection is here.
10:15Breaks the data into the small blocks
10:18and hashing. So, there is a one bonus
10:21point for you.
10:24Interview will ask
10:26to you that which algorithm
10:30DDB used to identify the chunks or
10:33blocks? So, so this is the hash
10:35algorithm.
10:37Okay. Hash algorithm are using by the
10:40DDB to identify or removing the data
10:43from
10:44from I mean, at the time of backup
10:46operation from the data itself.
10:49Check the DDB for hash match. New hash.
10:53It stored the blocks and index metadata.
10:56For the matching one, update the
10:57metadata only.
10:59That means update the metadata only.
11:01That means let's an example,
11:03this is the data one which is created on
11:07Monday
11:08and on Tuesday
11:11someone modified this D1.
11:14D1
11:15dash.
11:16So, it only update the metadata only.
11:19Not update
11:20the all data. I mean, it's a let's
11:23assume it's 2 GB.
11:25And the changes will happen only 0.5 GB.
11:28The final size is 2.5.
11:31So, it's only make the changes about the
11:332.5 Sorry, 0.5 not for the all two to
11:37zero.
11:39I hope you are
11:41understand my point.
11:43There's a two word, burning and ceiling.
11:45So, I will cover the burning Burning is
11:47not itself a
11:495 to 10 minutes lecture. I will explain
11:51the next coming video.
11:53Okay, ceiling you can
11:54I will explain, but you if you able to
11:57identify what is the DDB ceiling, then
11:59it's So,
12:01helpful for your learning purpose as
12:02well.
12:04Okay, the next slide.
12:07What is that?
12:10Data chunk and hashing. Backup data into
12:13the split into the fixed size block.
12:15Okay, there's one more question for you.
12:17What is the fixed size block in DDB for
12:21I mean in DDB. So,
12:24fixed that is 128 KB. And
12:27this is a default one. If interview ask
12:29you what is the default value of
12:31um blocks in DDB,
12:34then you can tell them that 128 KB.
12:38You can check for the
12:40512
12:41KB as well.
12:43Each blocks is hashed using the
12:45fingerprint algorithm.
12:47Hashed
12:49using a fingerprint algorithm SHA-256.
12:51You can easily if you are working in the
12:53company environment, then you can easily
12:55check which algorithm are using.
12:56Generally it's using hash.
12:58The hash represent the blocks uniquely
12:59identify.
13:01The second step is the hash comparison.
13:03The hash is checked against the DDB.
13:06That is in this DDB
13:09DDB one, DDB two.
13:12I mean, the data is present in there or
13:15not. If it's present, then it's update
13:17only the metadata. If it not present,
13:19then it's
13:21put it. Block is stored in the disk
13:23library and index in the DDB.
13:26If duplicate, only metadata is updated.
13:30If there isn't duplication,
13:32then
13:33only metadata is updated. No data is
13:35stored again and again.
13:37If it is stored again and again, then
13:39the storage capacity is
13:41increasingly very bad. I mean,
13:43increasingly very high.
13:44DDB indexing.
13:46DDB maintains the mapping of hash values
13:48to the physical storage locations.
13:51Okay, there's a your It's talking It's
13:54talking about your storage location.
13:56Metadata is stored in a SID files. You
13:59can remember these files index cache
14:01folder.
14:02This enables a fast lookup and efficient
14:05space uses. I hope you come to the next
14:08one. Deduplication modes. Okay. Maybe
14:11the interview will ask in the your
14:13interview.
14:14Okay, tell me what are the modes
14:17available in our Commvault if you're
14:19using the DDB. So, there's a two client
14:22side deduplication, media agent side
14:24deduplication. So, from the name itself,
14:27all the work on done on the client side.
14:30Client side, that means before sending
14:31the data to the media agent. Okay, let's
14:35an example, the client C1.
14:37And here's the media agent.
14:40So, all the work done on the client
14:43That means before
14:46sending the data
14:48before sending the data
14:53to media agent.
14:55Before sending the data to the media
14:56agent.
14:58All the works, hash and comparison
15:00happen on the client before sending the
15:02data.
15:03And what is the media agent side?
15:05That means DDB is enabled on the client
15:08side.
15:10It enabled on the client side.
15:12Data is sent to the media agent which
15:15perform the deduplication.
15:17Where data is sent to the media agent. I
15:20mean,
15:21if you call if you recall in one word
15:24before sending the data and after
15:26sending the data. Okay?
15:29Before and after sending the data.
15:32Okay.
15:33Let's jump to the next part that is a
15:35pruning. I will
15:36cover the little part only here.
15:39Periodic cleanup of unused are the age
15:41out has entries. I mean, it will delete
15:44the unused data after age out. Let's
15:48example seven. There's a seven days
15:50retention period. So, data will delete
15:52after seven days.
15:54Free of the space and keep the DDB
15:56efficiency efficient trigger the
15:58retention policies as and the space
16:00threshold as well.
16:03Okay. This is
16:04There's a little about the DDB. I will
16:06cover
16:07pruning, sealing of DDB, what are the
16:09steps to create the DDB
16:12in the coming lecture, in the coming
16:14video. Okay, let's jump to the next
16:16slide
16:17and go through
16:19the questions which may be asked your
16:21interview. Okay? Your comment as well. I
16:24will reply your comment.
16:26Okay? So, there's multiple questions may
16:29be asked by the interviewer. That is
16:32What is DDB partition? Why it is used?
16:36Explain the the the first question
16:39generally asked. I mean, not included
16:41this one that is What is
16:43DDB?
16:46Okay? What is DDB?
16:49Explain the difference between client
16:50side and media agent side applications.
16:52I already explained how does a Commvault
16:54identify the duplicate data blocks
16:56during the backup operation, during the
16:57backup or the backup operation. I
16:59already explained.
17:01How do you size DDB for many media
17:03agents? You can work on it.
17:05What steps you would take if the DDB
17:07goes offline or corrupted? Okay, let me
17:10explain this one.
17:11If you
17:12If you're going for the troubleshooting,
17:13then first you need to check the logs.
17:17Okay? You can comment me which logs we
17:19need to
17:21check. Which logs we need to check.
17:25We need to
17:27check.
17:29Okay?
17:30What happens during the DDB migration
17:33and how do you prepare for it?
17:36Okay? What are the components of a DDB
17:39and how they stored? I already
17:40explained. How does Commvault handle DDB
17:43replication across the site? This is
17:45that technical level question. What logs
17:47or tools to use to troubleshoot DDB
17:50performance issues?
17:53What are the risk of running multiple
17:55DDBs in a single media agent? This is a
17:58very important questions.
17:59It is It is This is the license level
18:01question, okay? What are the risk of
18:03running multiple DDBs on a single media
18:05agent? So, there's a multiple issues.
18:07There's a multiple If you you can clear
18:09correlate that is the load balancer.
18:13Okay?
18:14Okay, guys? If you have any questions,
18:16if you have any doubts, then please ping
18:17me in the comment. I will I try I will
18:19try to explain your queries as well as
18:22soon as possible. Okay? If you have any
18:24questions, any questions regarding the
18:26DDB.
18:27Thanks for watching. Let's meet in the
18:30upcoming videos. Thank you again.

About this transcript

This page contains the full transcript of Deduplication (DDB) – Interview-Focused #commvault #interview #cloudcomputing #backup #technology by TechVault by abhi , generated from the public captions YouTube serves with the video. The transcript has 2,558 words across 499 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.