YT YouTube2Text

YouTube2Text → Transcripts → ARCHER2 - Containers for Reproducible Research: Introduction to Podman and Singularity - Session 3

ARCHER2 - Containers for Reproducible Research: Introduction to Podman and Singularity - Session 3 — Transcript

by ARCHER2 HPC · 9,151 words · 1,508 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:05Good morning everyone. So uh yeah thank
0:08you for joining
0:11today's course. Uh on Monday we learn uh
0:15about podman how to uh create uh
0:18containers how to use uh containers
0:21using a podman or well the
0:26it's a similar if we use uh docker so
0:30let's see the concepts that we have
0:31learned and the commands that we have
0:34learned are pretty similar just like a
0:36changing podman
0:39um yeah rather than typing podman and uh
0:42command docker the command that will be
0:45the same. Today we will see more about
0:48uh uh singularity
0:52uh as I yeah the lesson I share the
0:55lesson in the in the chat
0:58and I will um we'll go through it. Today
1:03we will use archer too. So uh if uh you
1:09have an account and you are able to log
1:12in you're ready if not let me know and
1:15we we can find a find a a solution.
1:20So what I will do is I will share the
1:24yeah the website.
1:29Let me see if I can possibly close the
1:33this.
1:37Yeah.
1:39And
1:42no, I want to show is this one. Yeah.
1:52Make this bigger. Yeah.
1:57This
2:03I yeah hopefully you can see the the
2:06website. So uh this is uh the course
2:09material for the second day and this is
2:12what we will cover. Uh the structure
2:16will be in the in the morning. Uh we
2:18will see about singularity and uh how to
2:24uh play with that on archer 2 and in the
2:27afternoon will be two uh will be like a
2:30well two exercises for you to uh follow
2:37and also the afternoon will be for you
2:39to ask a questions or if you have any
2:43anything that you would like to do. So
2:45will be like a more self pace. I'll be
2:49around but uh yeah will be like only
2:53well only will be exercises. The morning
2:56will be more uh well theory like how to
2:59use singularity
3:02on archer 2.
3:06So yeah, I guess that we can uh
3:10start here is explain a bit about uh
3:14uh what is singularity. Uh the idea of
3:18sing uh singularity uh is like a similar
3:23to u
3:25uh uh docker or podman but is more uh is
3:30like a more target to uh HPC systems.
3:35uh more target because it's more uh they
3:39have like some uh security uh features
3:44that uh are not in
3:47podman and docker. I don't mean that
3:51using a I mean in your in your laptop is
3:55safe to use podman or docker but on a
4:00share res resorts on a HPC system like a
4:03archer 2 series or any other
4:06supercomputers that you may have access
4:09that are some characteristics that uh
4:12yeah are not like suitable. We will see
4:16later today why uh one of the reasons
4:19for instance if you remember the first
4:21day when we check uh when we are in the
4:26container and we use who am I h I get
4:30that I'm root rather than the user so
4:33that could be a bit uh
4:37complicated for when we use an HPC
4:40system. So singularity has like some
4:43features that make it usable for a sh in
4:48on a shared resource.
4:51Here is also explain a little bit of
4:53history of uh about the singularity or
4:57maybe you you hear about obtainer. Some
5:00people said why
5:03or you can you may find why is not use
5:06obtainer and is use singularity.
5:09Uh, Singularity started as an
5:11open-source uh uh
5:15uh project.
5:18the not a problem but a while ago
5:22uh was um in Singularity started with
5:26the two versions the open an open
5:30version that is a singularity C
5:34community edition and a commercial uh
5:37edition.
5:39So uh that was a bit uh yeah not let's
5:43say were not tricky was not like maybe
5:47not following the
5:50um
5:52yeah was like yeah creating like two
5:55versions one commercial one that was not
5:58commercial. So there were some concerns
6:00because this yeah the this was an open
6:04source project and but it's not let's
6:06say it's not the first time that this uh
6:09happens and uh what uh but um the
6:15solution was to uh what solution like
6:19obtainer was a let's say a fork of that
6:23project. So obtainer is a open open
6:27source and follow the same uh the same
6:30but in the for instance on archer 2 we
6:33have install singularity the community
6:36edition if you use sirus sirus is
6:40obtainer but uh
6:43doesn't change anything the features
6:45will be the same like yes like yeah well
6:49not yesterday two days ago again that we
6:52saw a OD man uh uh docker so will be
6:57kind of uh similar
7:02uh okay so now we will start with to
7:05check how singularity singularity works
7:09so we will log to uh archer 2 uh I will
7:14make yeah give me a moment Find
7:16something.
7:40I will look into. So, okay. I I need to
7:44stop sharing. Um share my screen.
7:54No, I don't want to share that.
7:58I want to share this
8:03to make the font bigger.
8:08So uh
8:11yeah there are as like the first day
8:13there are like the option of following
8:15the
8:17the exercises uh well following the
8:21replicating the what I'm typing on the
8:25on the window or you just can follow the
8:29as I said also first day that this uh
8:32this will be recorded so you can go on
8:34your own page in case that uh
8:38you want to uh yeah
8:41replicate what we are doing uh uh today.
8:46So I'm uh here I well I created a folder
8:50called singularity and I'm in that uh
8:53folder
8:56actually I can be
9:03yeah we can use that because uh uh this
9:06folder if I oops if we type here where
9:09we are we are in the work uh directory
9:12but it's not uh actually we switch to
9:14home directory Sorry,
9:19I will create a folder called here
9:21singularity or actually you can call it
9:23a container. So,
9:30and now I'm here in a home. I'm not sure
9:34if you are familiar with Archer 2 that
9:37in Archer 2 we have a two partitions. We
9:39have a SLHOM and SLwork. If we want to
9:44submit something to the compute node, we
9:47will need to move to slash work. We will
9:50do that later
9:52um
9:54in the afternoon. Also a heads up that
9:57uh because of what you may be aware that
10:00uh in the UK there is
10:04heat wave. So uh part of Archer 2 is not
10:08uh Archer 2 is running at a reduced uh
10:12capabilities not uh I think it's now 25%
10:16of the machine. So my recommendation is
10:19to use the short Q. If you use the
10:22standard Q you may see for a while on
10:25the the the queue.
10:28You see that this is because I'm using
10:30the my staff account se. But if you type
10:35a PWD, you must get a TA
10:3923
10:42six maybe or yeah the the number of your
10:46of the the train the training project.
10:50Any questions so far? Everyone is uh
10:54ready to keep going. Uh
10:57any issues or logging in or
11:04Hopefully not. If uh if so, just let me
11:08know. I'm happy to. Yeah.
11:13Okay. So, uh let's continue. Uh so yeah,
11:18we log into uh Archer 2 and uh
11:22if uh
11:25not in but in other supercomputers you
11:28may need to do a module load singularity
11:32but in our on Archer 2 is already loaded
11:35by by default. Singularity is already
11:39available by default. So to check if a
11:41singularity is installed
11:44you can uh do singularity minus minus
11:48version
11:50and see the the version of that we are
11:53using that is uh 373
11:561.
12:02Okay. So uh oops this is what uh we have
12:06uh
12:08now uh well here in the documentation
12:12there is like a reminder about
12:14singularity image and sorry singularity
12:18and the difference between image a
12:19container uh I think must be worth
12:23repeating it again the image is the
12:28u
12:30how to say the is kind of the template
12:33that you use every time that you run a
12:36or you create a container is like the
12:38base that you use the that image and
12:42when you run the image you get a
12:45container.
12:47So let's say the image will be kind of a
12:50let's say a template or the the the
12:54starting point and uh
12:58yeah when you run the image you will get
13:00a a container.
13:02Uh we can see that with an example.
13:07Uh one
13:09what we will do is we will uh uh use uh
13:14we will uh download uh
13:18uh
13:20image that is called well yeah the name
13:23is a bit lo cow
13:27to download that image is a singularity
13:32pool uh the name of image that will be
13:36code si s i
13:41and then the address will be library
13:45uh
13:47column sl
13:54one thing that I haven't done
13:57a moment
14:10I haven't run this command before.
14:16Well, if I will run that.
14:23Oh, well, okay. Actually,
14:25lo.
14:27Yeah, fine. Okay. So, yeah. Uh,
14:33so we get that the image. Okay. Since
14:36that is the biggest 90 mgabytes. So, if
14:39we do S, we have there the image. If we
14:42do L minus L H,
14:45we will get that. Yeah. 91 megabytes.
14:51So this is to download the the image and
14:56to uh to run it. Uh that are like that
15:00are let me see if I
15:10uh to run it there are like three ways.
15:13We will start with the first uh one that
15:17will be singularity run. So if we type a
15:20singularity
15:22run uh l cow sif
15:29you will get
15:32uh
15:34yeah it's a cow. Yeah, sometimes when I
15:36look it looks like a donkey but what
15:40cow that that tells you the
15:43yeah the time that I we are now
15:48um yeah tell you that with the funny
15:53colors.
15:56So so again uh what what happens when we
16:00do that uh we run the
16:05we start the the container, we run it
16:09and we close it. So this is like the the
16:13workflow that we learned uh on Monday
16:16that this is what happens when we uh
16:21when we uh run a
16:25a container.
16:29Okay. What was uh next? Uh when we run
16:34the
16:35the container by default use the
16:41use the the command that is like
16:44if you remember from Monday we have a
16:47cmd
16:49that this is the command that you run by
16:51default. And for these images you can
16:54check that with singularity. You can do
16:57singularity inspect
17:00inspect
17:02uh r uh l cow.
17:08And you see that the script that uh runs
17:11is a date
17:14cow say
17:16this.
17:19If we run that on our uh
17:25but if we we run that
17:35you see that these commands are not on
17:38archer 2. Why? Because these commands
17:40are on the container and this is what is
17:43run internally every time that I
17:46uh run this uh image. Any
17:54questions uh so far
17:59or uh clear?
18:10If so, we can keep uh going.
18:19Okay.
18:25So uh we we we have learned like a one
18:29way to uh
18:35uh one way to uh
18:40uh to run the uh container that this is
18:43only well using run with the run you uh
18:46run the default operation of the
18:48container and that's it. There is
18:51another way uh that is uh
18:57uh
19:00use uh exact. So uh you uh rather than
19:03running the default command rather than
19:05running this uh you can say
19:09uh
19:12you can specify what the container will
19:15do. So if I do singularity
19:21a moment but I yeah singularity act like
19:30you can do uh slashb echo.
19:45So here rather than using run we use
19:47exec. So I need to specify the name of
19:51the container and this is the operation
19:54that I'm uh running.
19:57Also here you can see
20:00for instance uh you can run this
20:02partially and see what uh uh caus.
20:10So uh
20:14for instance uh if I do
20:18day
20:25oops
20:32since That's it.
20:44Okay, that's interesting. I wanted to uh
20:49execute that, but it seems that is not
20:51uh
20:56okay. I
20:58I don't know what is uh
21:02probably you need to
21:05let's see if that works.
21:18Okay, maybe we can do this the with the
21:21third option.
21:23uh third option will be
21:27singularity shell uh l cow. So in that
21:31case you see that the prompt change and
21:35rather than having the outer to prompt
21:37let's say we have the singularity
21:40prompt.
21:42So here if we type a data
21:46uh code say
21:50we see yeah okay here it uh it works not
21:53sure why it's not working when I do x
22:00uh
22:02yeah I print the code saying that
22:05actually I can do a
22:08let's see if that works
22:19Yeah. So uh what to uh so calc what uh
22:22you give is the yeah the message that uh
22:28and I think low cut gives uh yeah the
22:33colors of the
22:37uh
22:39you can see which shape
22:45I'll find the word.
22:48Yeah, maybe we need to use like the full
22:51product. What is
22:55So, this is the
22:57Let's see the three ways that we can uh
23:01uh use
23:04when we um
23:08when we uh use um
23:13when we use a a image on on our two Any
23:25any questions so far?
23:51If not, yeah, we can continue.
24:04Oh, yeah.
24:09uh
24:13so uh this is the this is the image this
24:17is the let's see the container here uh
24:21for uh
24:25entry to the first day was pman we have
24:29a
24:32yeah this file is the the image The
24:35container image
24:38that is for singularity the is sif
24:44is the extension.
24:48So yeah, it's like the Yeah, singularity
24:51file maybe is the right.
24:56Let me check it.
25:06A singularity image format is the
25:14so uh to uh
25:17yeah to uh yeah there's a question in
25:19the chat to visualize the container uh
25:22we can do that with the we have three
25:25ways uh just to recap three ways to run
25:28the container uh we can do a sing
25:32singularity
25:35run
25:37um the name of the container that is
25:40this one. So if we do that
25:44we run the container and we print this
25:47uh value.
25:49What is the other way? Rather than using
25:52run we will use as that uh that means
25:56that uh rather than using the standard
26:00uh command that is in
26:04if
26:05the standard command is uh rather than
26:09running this
26:11we will run something else. So we do
26:13singularity except
26:17uh we can print uh
26:44Not sure what's happening here.
26:51This is not the output that master.
27:06Okay, let's just start taking I think
27:07something.
27:09So singularity run
27:12cow.
27:14So we got this. If I we do singularity
27:17exact
27:20uh cow and say
27:24in echo
28:11Okay. It seems that for some reason this
28:13is like not working the
28:16uh okay the yeah for uh for the
28:19container we have an ls but here the
28:23container will be save as a file in
28:25order that okay that that's a good
28:27question the way of working is that if
28:30we do ls we will have the the image here
28:34so when we download it on on podman or
28:37docker will be uh save somewhere
28:41So we'll be like in a internal
28:44directory.
28:46So we don't have a an ls on uh
28:50on for singularity singularity what it
28:52change that yeah the image will be in
28:54your folder
28:56for instance we will yeah just with ls
29:01we have this one and
29:04yeah that's it there.
29:08Yeah, for some reason when I use like
29:10this uh exclamation
29:13is not running it's running a bit funny
29:16but I think maybe
29:24Oh yeah okay if I use single quotation
29:26marks
29:28yeah it will work.
29:32Okay, so this is the second and the
29:34third way is like using singularity cell
29:37and the name of the image
29:40and then I will have a prompt and here I
29:43can check
29:45ls and dwd
29:49etc.
29:53Well later we will see yeah now well we
29:56see that the
29:58the difference here if you remember with
30:01the podman or docker when we do ls we
30:04were in the root
30:06in the root directory here I'm in a home
30:11directory
30:15because this is a this is something that
30:18is
30:22um
30:25Yeah, that by default uh singularity uh
30:29mount home directory we will learn that
30:34later
30:36later I'll see what later we will see
30:39the differences
30:42any questions
30:45so far
30:51if not The next step is we will uh
30:57we download this uh image but uh we will
31:00see how we can use docker to download an
31:04image.
31:07So uh you may remember that for uh
31:14for doc uh for singularity to get an
31:17image we use a singularity pool
31:22and specify the name of the image that
31:25well now we will uh use python so
31:31let's use this one for instance and I
31:34need to specify the address that will be
31:36docker per column SL
31:41Python and the tag that will be 3.14
31:473
31:50I will get like a reduced version of it
31:59interesting yeah command of found
32:00because I you see that yeah I'm inside
32:03the
32:04still inside the container so I will
32:06need to exit
32:09and be repeat the same. I will copy and
32:11paste.
32:24Now here you will see that uh you will
32:27get something because I'm uh using cash
32:31saf image. This is because uh yesterday
32:35was playing with that and this uh image
32:37is already on the cache. Uh
32:42if we want to we can do singularity
32:47cache clean
32:58and this will delete all the cash. So
33:00now if we pull the image
33:05in our
33:13you will get all of this input well
33:16output of
33:21so it's downloading the image and
33:23converting that to a singularity image
33:26file. So if we do a l now I have a two
33:30images. I don't need to uh uh we don't
33:33need to run the command uh uh pman or
33:38docker container ls or image ls. Uh well
33:43that will be image ls. Uh the cont the
33:46images will be on our folder.
33:52Now let's see if what happens if I do
33:54singularity
33:57run
33:59um this image
34:03I will get uh automatically I will get
34:06uh
34:08uh the prompt will change and will be a
34:10Python. So I can run
34:21this Python command.
34:24Oh, I don't know if it was.
34:35Yeah, I can do some operations with the
34:42I'm not sure if this this I don't think
34:46that this will work, but
34:49yeah, the version is not defined. So, uh
34:53yeah. So this is like I got a
34:56Python. So this is useful for instance
34:59if uh because on archer 2 there are uh
35:02several versions of python.
35:06I don't know if there is a
35:10well for instance we don't have python I
35:12think we need to do module
35:15load create python.
35:21So if we will do that
35:25we get on archer 2 is install Python
35:28here using this module and we get Python
35:32the version 3.10.
35:34So uh if we want to use another version
35:37of Python, we can either download
35:40the and install in our Python with the
35:46mini cond
35:48or we can use a a container.
35:54Okay. Yeah. Sorry I I haven't explained
35:56that on uh Archer 2. when you log to
35:59archer 2 uh there is some there are some
36:04software that is already available that
36:07uh if you do modular
36:11a
36:13you will see the software that is
36:14available
36:17so to you have
36:20uh all of these uh modules
36:25so uh well these
36:29Let me check.
36:32Yeah, this is like related to Python.
36:35These are libraries and this is
36:38software.
36:40We have caster co c2k grow max quantum
36:44tracer. This is already installed on
36:46archer 2 but it's not uh
36:50loaded.
36:52So if we want to use any of this
36:55software, we will need it to load that
36:58to the well we have like a more so this
37:01is like the full list of software that
37:03we have.
37:06Uh if I do module uh list
37:10this is the modules let's see the
37:13software that is already that are
37:15already load by default you will have
37:1814. So you will have this ones already
37:22loaded. I work this towards that. What I
37:25did was to load this uh module here. If
37:30I don't want to use it anymore, I can do
37:33module rm and create python.
37:39So now if I do module list, you see that
37:43this module is no longer here. So
37:46modules are like a way for when we use
37:51a a remote system to load the software
37:54that we want to use or not. That's why
37:57at the beginning I said that we can do
38:00module in some uh
38:04remote systems we can do module load uh
38:07singularity because uh by default is not
38:10loaded.
38:12Actually I don't know if that will work.
38:17Oh well, if I type something that is not
38:19uh doesn't exist. Yeah, we will
38:23complain. There is not a module called
38:26singularity because singularity is
38:28already. So yeah, this is the so yeah I
38:31sorry I haven't introduced let's see the
38:33module terminology.
38:51Okay. Any questions uh
38:58so far?
39:10If not uh well I already have a clean
39:14that but uh is uh
39:24yeah let's see about like the cache that
39:28uh when if I uh we have here two
39:32containers uh this l cow and python If I
39:35delete the Python container with the RM,
39:39also be careful with when you use that
39:41on a terminal on an arro too because RM
39:46will delete and will not be a
39:50recycle bin or will not be like a way of
39:53to recover the file that this is deleted
39:56is gone.
39:58So if you do python
40:01rm python actually also when you use rm
40:06if you want to be sure it's not
40:08recommended to use it on a script but if
40:11uh you want to include it you can uh to
40:16be sure use the option minus i. That
40:19option will ask you uh do you really
40:22want to remove regular file and
40:26depending of the answer if you answer no
40:29it's fine
40:31you will still have the file
40:33but if you answer yes or why you will
40:37delete the file
40:40now if I do
40:43oops uh
40:47so what commands that might be use for
40:50you to clear will clean the terminal and
40:54also if what I use a lot is a control R
40:58that this is a reverse search. So if you
41:01type now pool
41:04will show you the last uh entry of uh on
41:09on archer 2 that that has a pool in the
41:14because I want to do singularity pool
41:16python. I just press enter and I don't
41:21I don't need to uh type the whole
41:24command. We'll use uh uh the latest
41:29command that uh pull is in uh on the
41:33command line
41:36and here I have get info using using
41:39cash uh as I have image.
41:44Uh
41:48this is because uh the this is is on the
41:52cash. If we do singularity cash
41:56list
41:59minus B,
42:01you see that they get um
42:12uh
42:15I got a container file and I use a 40
42:19mgaby and six OC blob files. These are
42:22the docker files that these first we
42:25download this uh these are the layers
42:28that we saw like the first day we
42:30download them and then we created the
42:32container.
42:35So you see that the layers will work
42:37well. They said that this 40 megabytes
42:40and the containers itself is 40 megaby
42:44as well. While total space use is uh 81
42:50megaby or well will be close to 82
42:53megabytes.
42:57I think we have also this option that we
43:00can check
43:03similar to docker or uh podman. If we
43:08type a minus minus help
43:11uh gives you like a list your local
43:13singularity cash
43:17and get minus v the verbos
43:24list of cash. Well, you can specify here
43:27the
43:30the types that by default is all.
43:35Oh, yeah. These are the options that we
43:37can use.
43:50Also that by default, let me check if
43:53this is a
43:58because I think it's not uh
44:05yes notify
44:07uh by default all of this uh cash is
44:11saved in uh
44:13uh home that if you want to see home is
44:18this my directory my personal directory
44:20here so it says save in uh
44:25home/
44:28singularity
44:33cache.
44:35So if I do ls
44:38you will see that I have here several
44:40directories and if I go to uh
44:45C
44:47this will
44:49tmp
44:51here you see that I have what like uh
44:54this will be the image that I have or I
44:57can think it was low
45:01yeah have Uh
45:06yeah,
45:07this is the the cache that uh
45:13yeah, here we have the the layers that
45:16uh I was talking before.
45:20So if we set this uh singularity cast to
45:25uh we can change the location.
45:30Uh the cars will be saved uh somewhere
45:34else and the way to clean the cash is
45:37what is
45:39playing the documentation is
45:41singularity.
45:42Actually we can check with this command
45:46d minus sh
45:48/
45:50do singularity
45:54uh cach
45:59you see that directory is 73 mgabytes.
46:02If we do singularity cash
46:07clean
46:11uh want to continue
46:16so yeah has been removed this this this
46:20we don't need to remove anything library
46:24we remove this entry and nothing to
46:26refine so now if I repeat again using
46:31the
46:32up arrow I can check previous commands.
46:36So if I repeat that
46:39you see that now the size rather than 83
46:42mgabytes is a 32 kilob.
46:53Any
46:56questions or
47:01with that
47:37Okay. So, if not, we'll uh
47:41continue. I guess we can Yeah, maybe
47:45we can start uh
47:48yeah, we can have a maybe a small break
47:51uh at 11. But yeah, I can start the next
47:57lesson.
48:04Yeah, let's do that. Uh
48:08so we have seen okay we do a and we see
48:11that we have these two containers. We
48:14will work u
48:16actually yeah we can work with this
48:18container with the Python one just to
48:21for a change u we can do singularity
48:27shell
48:28and python
48:32and now I'm in the
48:35in the container using this uh python
48:37image
48:39uh before we have a grow of who am Hi.
48:44And I'm uh you will get your username.
48:47My username in this case is J.R. Rodri.
48:52And you may get TA
48:55on your yeah some character characters
48:59from your name.
49:02You also might uh have uh if you do uh
49:08pwd
49:10that I'm in this uh directory
49:14I'm not in the in the
49:18on root but what I can change to root
49:22and dos
49:23actually I think
49:26s minus l to show the
49:34This is one thing that is done in the
49:39uh so uh singularity
49:42is used on HPC systems rather than
49:45docker or podman uh because of this uh
49:48that is like a more adapted to remote
49:50systems. I when I uh by default rather
49:54than using the root username use my
49:58username
50:00also by default mounts the
50:06for instance these folders being are
50:08mount to this uh u us
50:15this is mount to the the folders that
50:18are in the in the host most uh OS if I
50:23move to home
50:28and it will
50:38it's not showing the that on the
50:45and also mount the uh there are some
50:49directories that are mount
50:51mounts the
50:55takes my username because mount the
50:57slftc
50:59password is the like the file where all
51:02the
51:03users on the machine are.
51:07So if I do I will do well exit and
51:10actually will share my
51:13the yeah
51:18the documentation.
51:22No I don't want to mute or to stop
51:24sharing.
51:35I will make this small.
51:43So we have a singularity container here
51:45that is on our uh uh on the right of the
51:49screen and the host that is on the left
51:51of the screen. So uh home
51:57by default is uh available my home
52:00directory is available on the container.
52:03We don't need to mount if you remember
52:05that with the docker of podman we need
52:08to mount our directory but here is a
52:11mount by default
52:13and the
52:15use my right username. So if I create a
52:19file with a container on my home
52:21directory, it will create it with the
52:24right user with the right permissions
52:27and uh
52:30will be like will be I mean easier will
52:33save us to mount on mount
52:37uh directories.
52:42Not sure if that uh
52:44makes sense. What we will do now is we
52:48have a this uh exercise. I will copy
52:51that in the chat in case that uh you
52:53want to have a look.
52:59So uh
53:02what we will do is we will uh well you
53:05can have a look at the exercise and I I
53:07will give now a uh 15 minute break. So
53:12we'll be back at uh uh quarter past 11
53:18and we will keep uh working with
53:23with this and also we will learn learn
53:26the other
53:28uses that we can do with the with the
53:30containers as I said in the afternoon
53:33will be all exercises like we will use
53:37also the
53:39the submission Q that we want to submit
53:42jobs to uh to the queue to archer 2 to
53:46run them on yeah the compute nodes
53:50and uh yeah that's it also I'll be
53:53around if you have any questions but
53:55yeah now will be
53:57uh break time I will
54:00okay hope you had a nice uh break you
54:03have any questions please uh feel free
54:05to post them in the chat if not I will
54:08uh continue with the
54:15with the le
54:29I okay
54:45So if I
54:49if I open a
54:52a if I create a container with the
54:55singularity share,
54:59you see again that my prompt change if
55:02you do LS minus no ls sound
55:07or ls minus l h
55:12you will see that uh
55:15files will get the right so the files in
55:18the on the
55:20slash will have a will be root
55:25and for instance home will be my
55:27username and the group that that I
55:33So if I want to like create a file uh
55:37you can create with the touch slash uh
55:41princess my file txt
55:46it will fail because this is a read only
55:50file system I don't have a permissions
55:52to create that file on the
55:57but if I want to create the file on uh
56:00if I So if I do CD,
56:04I'll be at home. So I can go to CD
56:07containers
56:09and I can here I can create a file.
56:13Um
56:19and I have my file there. And if I exit
56:23and I do ls, my file will be there. And
56:27if I do ls minus l you see that the
56:31permissions are okay. I have uh the file
56:35belongs to my username and the file
56:37belongs to my group that is uh J Rodri
56:42set 19.
56:47Is this uh
56:51clear?
56:58So as I said by default I have like the
57:01home uh directory mount but what if I
57:04want to mount another directory for
57:07instance I said that we have a uh home
57:11that this is like a /home
57:15twice the group and then my username.
57:19Uh
57:20for instance, we can try to mount work.
57:23W work work will be SL work 719
57:28718 and my username.
57:35So I can try to do that. So to do that
57:38will be uh singularity
57:42cell and uh you can specify minus uh
57:47capital B and the
57:52the directory that you want to mount. So
57:55will be /work 79
57:5979
58:01and my username
58:04in your case. I don't need to in your
58:07case will be like a /work t I think it's
58:11a 236
58:14twice and then your username
58:17and now I can yeah I can specify the
58:22name of the image.
58:25So if I do that and I do ls
58:29minus lh
58:37you see that I have here
58:39uh the work directory so I can do uh cd
58:43/ work
58:48and I will have my directory uh here
58:53if I do s I get this directories I I can
58:58I can use it
59:00on my on my container to build the
59:03software to run something
59:06uh
59:11if I do exit
59:14if I don't specify anything else this
59:17will mount the same uh but uh if I want
59:20to change for instance I will I want to
59:23mount it and uh
59:26just uh like using a slash work rather
59:28than using the whole uh thing I will use
59:32a semicolon
59:33and the name of the the name of the
59:36directory in the in the image.
59:40So if I do this and I do cd / work
59:46ds
59:48I get uh
59:51this directory
59:56map here in the in the on the container
1:00:01I can do the same I can add other
1:00:05directories as well. For instance, if I
1:00:08we will see later in the with the with
1:00:10the exercise that if you want to use the
1:00:14Cray compiler. So the CR uh you can
1:00:18mount for instance uh
1:00:24if you add a comma you can mount another
1:00:27directory. So you can mount for instance
1:00:31/ts/cra.
1:00:35So if you do that, you can go
1:00:38and you have here
1:00:43the directory that is on on archer 2 on
1:00:47your container.
1:00:52Any questions so far?
1:00:56Let me see if I'm forgetting something.
1:01:18If not, okay, let me check the
1:01:29Okay, let's have a look at this.
1:01:39These are other topics that I would like
1:01:41to uh share with you uh and are in
1:01:46uh so for instance with the I'm not sure
1:01:49if you are familiar with the environment
1:01:52variables that when we use on
1:01:56uh if we do export here on uh on the on
1:02:03archer 2
1:02:07we got a lot of variables here one
1:02:10variable for instance is uh home
1:02:16if I'm scrolling
1:02:18so this variable
1:02:23here oh I get uh also the this variable
1:02:27for instance host host name host type.
1:02:31So by default these variables are copied
1:02:36on the on singularity.
1:02:39Another way is to use M but with M.
1:02:44Yeah, you get also all of these
1:02:46variables.
1:02:48Actually let me check.
1:02:51So if I do singularity
1:02:55uh will be
1:02:58if you're wrong in set x
1:03:01uh python
1:03:03I don't know if export will work
1:03:07no m
1:03:13you see that all of these sorry if it is
1:03:16like a quite a long list all of the
1:03:18variables that are on archer two are
1:03:22copied to the to the to the
1:03:25will be using the container.
1:03:29But there is a way in if for some reason
1:03:32we don't want that to that to be
1:03:36that to be copied on the
1:03:40on the container I can do singularity x
1:03:45uh clean
1:03:49environment
1:03:50on this uh so if I do that
1:03:54if I add this you see that now the
1:03:57output is uh smaller because these
1:04:03these are the the environment variables
1:04:05for the machine are not the environment
1:04:07variables on on archer too.
1:04:10Uh I see that there's a question on the
1:04:12chat.
1:04:19That's Yeah. Okay. That's an interest.
1:04:22So the the option that uh we we have
1:04:25used minus B before
1:04:28this will be for bind console from a
1:04:31bind.
1:04:34So this is used when we want to bind a
1:04:37directory of the uh the machine or the
1:04:42host machine or archer 2 to the to the
1:04:46container. Minus v comes from uh verb
1:04:49boost that this is normally to uh print
1:04:53more
1:04:56uh
1:04:58to like a yeah print more output. So
1:05:01this is normally when you are debugging
1:05:03and you are doing something to see what
1:05:06is uh what's what is uh going on
1:05:11is that uh does it solve uh
1:05:17your question?
1:05:22Yeah, you said maybe I Yeah, maybe there
1:05:25there is software that use normal. Yeah,
1:05:28minus comes from verbose. But yeah, it
1:05:31could be the case that maybe but by
1:05:34default is yeah minus uh is like
1:05:40yeah that are like some uh it's like a
1:05:43minus h normally say use for help minus
1:05:47i is for input minus o for output. There
1:05:51are like some flags that uh are kind of
1:05:54I would not say standard but are used
1:05:59normally for uh
1:06:03uh for uh yeah for certain uh
1:06:07operations.
1:06:21So actually yeah that
1:06:28I mean essentially I mean maybe it's not
1:06:30but what we for this course are is the
1:06:35same is like a minus bind is like a is
1:06:39the mount there are like a several types
1:06:43but the one that we see here is the bind
1:06:46So will be
1:06:49I mean the
1:06:52will be the same is there is no I would
1:06:54say I use like a mount or bind like the
1:06:58I use two different words but the
1:07:01essentially are are the same
1:07:05but what we use I think there are other
1:07:08ways to
1:07:11mount. Yeah, exactly. There are uh but
1:07:14the types that we see in this lesson are
1:07:17the actually the types that I normally
1:07:19use is a bind is the but essentially is
1:07:24the kind of the same the thing with
1:07:26mount is
1:07:29mount if you use u but it's not for
1:07:32containers for operating system you can
1:07:35mount
1:07:37a disk or mount
1:07:40another file system but this Not let's
1:07:43see the purpose of so let's see what for
1:07:47what we are learning is the I would say
1:07:50is the same we do mount and the type
1:07:54that we use is let's see the bind
1:07:58but yeah essentially
1:08:01and yeah on singularity is minus uh if
1:08:06we do singularity
1:08:10uh
1:08:15I guess we here we have a
1:08:19yeah this is the full documentation of
1:08:22oh I forgot to mention that we can also
1:08:25mount uh the when we mount yeah or bind
1:08:31that uh we can specify if is a ro read
1:08:36only or rw that this is a with the right
1:08:43here. I was using clean environment but
1:08:45there is a shorter option that is minus
1:08:48uh e
1:08:51actually yeah you can use minus capital
1:08:53b or minus minus by in case that you
1:08:55want to uh
1:08:58uh make it uh
1:09:01say longer or
1:09:09also here we have a if you remember that
1:09:12we were using docker library. We were
1:09:14using Docker because this is a normally
1:09:17the the image if the image is host on
1:09:21Docker Hub, we will use Docker but if
1:09:25the images because also we have this uh
1:09:28repository
1:09:30from uh Silab that is the company that
1:09:33now runs Singularity. We can use a
1:09:38library
1:09:41or also we can use the directory. We
1:09:43will see now what is a sandbox
1:09:47and the singularity hub. Yeah, there are
1:09:50like a different uh it's always good to
1:09:53have a look at the documentation and see
1:09:54the different options because sometimes
1:09:56we can find the options that we can uh
1:10:00uh use.
1:10:09Uh
1:10:15okay. So I first bring uh the
1:10:27taking
1:10:35we don't have here the option
1:10:41before I was explaining about the the
1:10:42the environment
1:10:45cleaning environment before running
1:10:47container and also we have this option
1:10:49as environment variable to container
1:10:52processor.
1:10:53We can either use this
1:10:56minus I minus m and define the
1:10:59environment
1:11:01uh variable here
1:11:06or we can create a file with the
1:11:09environment variables and pass the the
1:11:12file.
1:11:18Uh how we can do that? Uh so for
1:11:20instance if I do a
1:11:25singularity
1:11:27cell
1:11:30Python
1:11:35and I echo
1:11:42for instance my name
1:11:45it will be empty.
1:11:51But they can do singularity
1:11:53cell - m
1:12:05for instance
1:12:08and if I do echo
1:12:10my name
1:12:13will print my my name. So this is a way
1:12:16in case that for some reason I want to
1:12:20uh set up an environment variable and
1:12:22send it to the container I can do this
1:12:24in that way
1:12:27also I can use a file. So a file uh
1:12:34I will copy this. A file could be
1:12:38uh you can use uh well to create a file
1:12:42on archer to you can use either vi or
1:12:44nano probably.
1:12:51Well nano in theory is easier.
1:12:54my bars.
1:12:58Let's see.
1:13:01So here I can copy
1:13:03the variables that I want to uh
1:13:07I can say my surname for instance.
1:13:12Yeah, this a bit silly example but it's
1:13:14to just to
1:13:16show you. So to save this, Ctrl O
1:13:20and enter and then Ctrl X. So now you
1:13:24can do a singularity
1:13:26cell
1:13:30I think was MV file.
1:13:33I go into the documentation. Yeah. M5
1:13:50bars txt
1:13:55uh and the
1:13:59and here I can do a call
1:14:03measure name
1:14:06and my name.
1:14:08So I have this uh list of environment
1:14:11variables.
1:14:14Maybe is the case that you will never
1:14:16use it or is not needed but well you
1:14:19know that this exist.
1:14:23There are different ways to also to send
1:14:26the variables but what this like we say
1:14:27the one that
1:14:30we can think also you can use uh
1:14:34um
1:14:40yeah but I would say this is like the
1:14:42more the most simple way to
1:14:45to do that.
1:14:51Okay. So this is about environment
1:14:55variables. I'm not sure if you were
1:14:57familiar with them or not.
1:15:01Uh
1:15:04another thing that will be useful to
1:15:08Okay, you see that here in my directory
1:15:10I have a two uh
1:15:15I have two uh
1:15:17images and what two files but I can
1:15:21let's say this is like this Python file
1:15:24is built on a
1:15:31it's uh built uh on a rece. So I can
1:15:35create uh what is called as a sandbox
1:15:39and uh you will see all of the files
1:15:41that are on the
1:15:44there. This is in case that for some
1:15:47reason you remember yesterday that for
1:15:50the container we were able to uh copy a
1:15:54file uh to the container and uh keep it
1:15:58there.
1:16:00uh we will try to do the same using the
1:16:03what is called a sandbox.
1:16:06So the way to do that
1:16:09I can use the Python. Yeah, let's use
1:16:11this Python image will be singularity
1:16:17build
1:16:19minus minus uh actually I can check I
1:16:22will check the help here.
1:16:29What is quite a lot of P
1:16:41So you can build like a sandbox that is
1:16:44a readr container with the like a
1:16:46directory uh a structure
1:16:50here is explaining the
1:16:53image you have all of these uh options.
1:17:02So singularity build
1:17:06minus minus sandbox
1:17:10I will need to specify the directory. So
1:17:13directory will be python and rather than
1:17:16use s if I will use uh emgd but you can
1:17:22use uh or even don't even use like a
1:17:26different name. This is like a just for
1:17:30to know that this folder belongs to that
1:17:34image.
1:17:36And now I specify the name of the image.
1:17:42Well, you get here like some warning
1:17:44info etc.
1:17:47And this is what what is doing is a
1:17:51compressing
1:17:53the
1:17:56the
1:17:59folder there in the so if we do now you
1:18:03will see that you have here the
1:18:09the image but also you have the image
1:18:12that will be in a listing in a folder in
1:18:15folder format. So if you go CD Python
1:18:18and you do ls, you will see that there
1:18:20you have all of the files that are on
1:18:22the
1:18:28like are on the on the image
1:18:36and you can uh
1:18:39draw the same as we have doing. If uh
1:18:42you remember we have done singularity
1:18:44run
1:18:45well let's go back to one directory
1:18:49we have done singularity
1:18:52run python
1:18:55sif
1:18:57and here I have python I can print
1:19:06you can do the same with the image Here
1:19:15you can do uh
1:19:25yeah print Okay.
1:19:42This is uh this is uh interesting in
1:19:45case that for some reason we want to
1:19:48like add
1:19:51a file to the
1:19:55in case that we want to add a file to
1:19:58the
1:20:00uh to the image.
1:20:09Uh for instance, we can do uh
1:20:13we can do an example and you see how uh
1:20:17um
1:20:22how to do that. This is a bit more
1:20:24complex but what the documentation I
1:20:26will tell you where where it is in
1:20:28there. So uh that is also a way that for
1:20:33yeah we don't build the okay let's try
1:20:36this and see if that so we have a
1:20:38singularity
1:20:40run
1:20:43uh I will need minus minus uh
1:20:48actually let's do
1:20:52I will need this option minus w
1:20:56I said by default all singularity
1:20:58containers are available as a read only.
1:21:00This option makes the file system
1:21:02accessible as a read write.
1:21:09So if we do that breakable
1:21:15and
1:21:22use the Python image here
1:21:34I don't want to use
1:21:38I want to is uh
1:21:41shor
1:21:49this.
1:21:51So I can create mk
1:22:09I can create here a file for instance
1:22:12saying uh
1:22:23hello. Let's see if uh I don't know if
1:22:26this will have a
1:22:32I don't know if this is here. No, it's
1:22:35not here either. Okay. Well, but the way
1:22:38to do that, I can do.
1:22:51So, I can create a file here that will
1:22:53be hello
1:22:55py.
1:23:00Yeah, since that doesn't look
1:23:11Okay, I will use a quotation single
1:23:14quotation marks
1:23:17unknown.
1:23:24Okay, so that works. If we do s we have
1:23:27this file here
1:23:29and if we use python hello
1:23:34it will print hello.
1:23:38I can yeah so uh for instance now if I
1:23:43exit
1:23:45the
1:23:47the container
1:23:49and uh this file must be uh there. So if
1:23:53we do if I we go Python
1:23:56we do /
1:23:59you see that the file is here
1:24:02and we have a this is because uh this is
1:24:05like a kind of a how to say yeah this is
1:24:07why it's called a sandbox when you uh
1:24:10when you exit the container the fi uh if
1:24:14you create some files there it will
1:24:17remain there it's not like a as you
1:24:20remember with the botman or docker
1:24:23When we finish the container, if we have
1:24:26created a file inside the container is
1:24:29gone, but it's not in this case. Also
1:24:31here we can edit the file and we can say
1:24:36hello.
1:24:37Oops.
1:24:50Incredible.
1:24:55Uh well also I didn't explain if you use
1:24:58a vi if you want to edit you need to
1:25:01type I and you see that the bottom
1:25:03changes to insert.
1:25:07So you can change let's use high. Now if
1:25:11when you press escape you see that this
1:25:13insert is a column and the way to save
1:25:16is a column w is a bright and q is a
1:25:21exit
1:25:23uh quit
1:25:26and this is a it's a vi could be a
1:25:28little bit tricky but uh is the one of
1:25:31the editors that are available in most
1:25:35systems. So if you feel uh when you
1:25:39learn a little bit about the vi this
1:25:41will be available everywhere. Actually
1:25:45the I think it's is not a VI it's a VI m
1:25:49the version that is available but it's a
1:25:52VI improve
1:25:54because the original version that is a
1:25:56VI is even more uh
1:26:00is even I would say more tricky to to
1:26:05use
1:26:07design when the like even when the
1:26:10keyboards they don't they didn't have
1:26:11the arrows so you would needed to move
1:26:14with the with some letters. Yeah, was a
1:26:17bit tricky.
1:26:20Okay.
1:26:21uh what I wanted to uh show. So if we do
1:26:28okay we are containers again have this
1:26:30uh so what I will do now is I will run
1:26:33the container doing singularity
1:26:38uh exact uh my container will be python
1:26:45no sh
1:26:51and uh I want to uh do python
1:26:56So opt
1:27:05and you see that it works. So we have
1:27:08singularity except this is the the image
1:27:12where the sandbox and I'm running
1:27:14python/op/hello.
1:27:17py. So the file that I have created in
1:27:20the sandbox remains. If I do the same
1:27:23with the original image, I will get an
1:27:27error.
1:27:29They said that
1:27:32cannot open file. No such file or
1:27:34directory
1:27:43and uh
1:27:48I'm trying to
1:27:57So this is a way I think the work
1:28:12I'm trying to check if something else
1:28:15Awesome.
1:28:34Yeah. Also, I'm not sure if because I
1:28:38haven't tried I think that when you have
1:28:40this sandbox, you can convert. Let me
1:28:43try and see if that uh
1:28:46if that works. Uh
1:28:51um
1:28:54okay. Singularity
1:28:56build.
1:28:58So what I can do is I can use now my uh
1:29:04I will say Python hello. This will be my
1:29:07image
1:29:09and uh I will use the sandbox. Uh the
1:29:13sandbox is a Python.
1:29:34So in theory now if I do singularity
1:29:36access
1:29:39using Python hello
1:29:56you will it will uh
1:29:59so a way also a way of building or what
1:30:03like Uh adding files to a image on a
1:30:07using singularity is like a you you
1:30:10uncompress the original image with
1:30:13sandbox
1:30:16you edit the files as you wish and then
1:30:20you do the you rebuild the no it's not
1:30:24access is here.
1:30:28So you do singularity build python
1:30:32hello. This is the new the name of the
1:30:34new image and the
1:30:38and the name of the directory that uh
1:30:41where the sandbox is and create the new
1:30:45image. Now if we do singularity
1:30:49cell
1:30:50python hello
1:30:53if we go to /
1:30:57we see that we have a hello
1:31:08any questions? This is a bit more
1:31:11complex but well this is like explaining
1:31:13if in case that you want to uh for some
1:31:16reason add some files or some scripts to
1:31:19your
1:31:21uh image. A way or if also if you want
1:31:25to have more control a way is to use a
1:31:29sandbox.
1:31:30And with the sandbox you can create a
1:31:33new image once that you're happy with
1:31:36the files that are in that sand sound
1:31:38sandbox.
1:31:56Okay. So, just to uh finish this this
1:32:00part
1:32:02actually let me
1:32:16Um
1:32:23there are some links here on the
1:32:24documentation. I will share them in the
1:32:26the chat. I will stop saying this and
1:32:30share.
1:32:46So one of the first they were they were
1:32:48ask question about the containers versus
1:32:50uh virtual machines. So you go here you
1:32:54have here
1:32:56uh some information about uh it is a
1:33:01smaller
1:33:03well it's a big enough the for the
1:33:05container architecture the virtual
1:33:07machine and also a comparison of
1:33:11uh virtual machine versus a container
1:33:16because operating system
1:33:20one main characteristic is that the
1:33:22operating system and the virtual machine
1:33:25runs a complete operating system
1:33:27including the kerner and it needs more
1:33:30resources
1:33:33and in a container runs a portion of
1:33:36that operating system.
1:33:46So this is one also here the container
1:33:48technologies docker pod m well maybe the
1:33:53state-ofthe-art is a bit uh not because
1:33:56this is like a YouTube video from
1:33:59so yeah probably not
1:34:04in case that for some you want you are
1:34:07like from what you are involved in
1:34:10bioinformatics research I hear also like
1:34:13some uh recommendations
1:34:16I will make this
1:34:20or how to uh
1:34:24well here for instance they advocated
1:34:26for one tool one containers
1:34:29Monday we saw that we can have a big
1:34:32container with several tools or like a
1:34:35smaller containers. Yeah, for instance,
1:34:38they define that is a one tool, one
1:34:41container
1:34:43because we have we haven't covered here
1:34:45in the that is like a what is called
1:34:51uh container container orchestration
1:34:54that is a workflow. So having several
1:34:57containers
1:34:58uh you can uh have a workflow. So
1:35:01imagine that you have a
1:35:04uh protein that you want to uh follow do
1:35:08some operations. So uh you use one
1:35:12container to do one operation then the
1:35:14output will be the input of the next
1:35:16container and so on. So we will create a
1:35:19pipeline of that
1:35:23and here there are more documentation
1:35:25about singularity obtainer and uh yeah
1:35:29this is the
1:35:32uh
1:35:34so what we will see this afternoon is we
1:35:37have a
1:35:40well two exercises. The first one will
1:35:42be singularity
1:35:44to run blast. that this is an example of
1:35:47uh of what we have seen to u
1:35:52uh we will see that without the having
1:35:54the software on archer 2 uh you can run
1:35:58uh
1:36:00this uh example
1:36:04and uh here we have a also but maybe you
1:36:07are not you are not familiar with NPI
1:36:11not the how to uh for instance how to
1:36:14build software or how to build an MPI uh
1:36:19uh code on a container and run it on
1:36:23Archer 2. This uh could do not be uh
1:36:27maybe it's not uh up to date. So my
1:36:29recommendation is to go to the to the to
1:36:32the docs the
1:36:35I will copy that on the that this is a
1:36:38update.
1:36:41So here if you go to uh use some best
1:36:44practice
1:36:46containers
1:36:48and then
1:36:53here using singularity with my pi on our
1:36:562 uh if you follow this uh tutorial you
1:37:01will uh so they yeah it's explain like a
1:37:04step by step uh here what you do is in
1:37:07your laptop you copy this docker file
1:37:10Okay.
1:37:12And you build the the image.
1:37:16Also, you need to specify here platform.
1:37:20And once that you build the image, you
1:37:22docker push. And when once that you are
1:37:26on archer 2, you use this uh running a
1:37:30script.
1:37:31Oops,
1:37:34sorry. That's something that
1:37:38use this running script.
1:37:40and uh you run out and if uh it runs
1:37:43okay you will see this uh output uh
1:37:49here.
1:37:50So this is what uh we will do after
1:37:55after uh lunch. Lunch will be like now
1:37:58the lunch break from uh from noon until
1:38:011:00 p.m. And uh after lunch will be
1:38:05like a yeah exercises and uh you playing
1:38:08with Podma or uh or Singularity
1:38:12containers on on Archer 2. So uh yeah
1:38:19please uh feel free to uh come and do
1:38:21the exercises but it's will be like kind
1:38:23of open. So uh if you want to uh yeah
1:38:28follow the exercises and have any
1:38:31questions uh feel free. If not uh or if
1:38:35you want to do them on your own uh you
1:38:39will have the account for I think for
1:38:41two more weeks.
1:38:44So uh you can do them uh in another time
1:38:48in case that this afternoon is not
1:38:50working well for you. Uh you can always
1:38:54get in touch by mail and we can see the
1:38:57my mail address is if you want
1:39:05so you can always get in touch. Anyway,
1:39:08yeah, let's have a lunch break and uh
1:39:12yeah, see you at 1 pm. So, any questions
1:39:15or anything that you may have, I'll be
1:39:17uh I'll be there. Uh yeah, bye for now
1:39:21and yeah, see you uh later today.

About this transcript

This page contains the full transcript of ARCHER2 - Containers for Reproducible Research: Introduction to Podman and Singularity - Session 3 by ARCHER2 HPC, generated from the public captions YouTube serves with the video. The transcript has 9,151 words across 1,508 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.