YouTube2Text

n8n Evaluation quickstart — Transcript

by n8n · 576 words · 82 segments · language en · Watch on YouTube

Full transcript

  1. 0:00Evaluation is a great way for getting
  2. 0:02confidence in your AI workflow. It's
  3. 0:04essentially a form of testing and it's
  4. 0:06the technique that helps you move from
  5. 0:07an impressive proof of concept to a
  6. 0:09solid production ready workflow. In
  7. 0:12fact, it's pretty much essential to do
  8. 0:14anything reliable with AI. And if you
  9. 0:16don't believe me, take these guys word
  10. 0:18for it. Anthropic calls it the crucial
  11. 0:20secret behind good prompting. And Vel
  12. 0:22goes as far as to say your prompt is
  13. 0:24broken if it doesn't have evaluations.
  14. 0:27It's important because LLMs can't be
  15. 0:29reasoned about like code can. So they
  16. 0:31have to be measured instead. Evaluation
  17. 0:34provides a way of doing that
  18. 0:35measurement. In evaluation, you run a
  19. 0:37test data set through your workflow to
  20. 0:39see how it performs on a range of
  21. 0:41different inputs. So let's look at an
  22. 0:44example. Here I am building a workflow
  23. 0:47for classifying support tickets. It
  24. 0:50takes an incoming ticket in the format
  25. 0:52of a subject and a body and using AI it
  26. 0:56calculates a category and a priority.
  27. 1:00I've got it working end to end and now I
  28. 1:02want to get a sense of how reliable it
  29. 1:04is. This Google sheet has test data that
  30. 1:07I can use for that. A handful of support
  31. 1:10questions ready for classification and
  32. 1:11I've also added in what the correct
  33. 1:13answers are by hand. By running them
  34. 1:17through my workflow, I can get a sense
  35. 1:20of where things are going wrong and get
  36. 1:22ideas for improvements. So, I've added a
  37. 1:24trigger here that will read in the rows
  38. 1:26from the data set and a set outputs node
  39. 1:29that will write the results back to that
  40. 1:31data set. Uh, and you can see that it's
  41. 1:34running multiple times in a row, once
  42. 1:36for each row in my data set. And we can
  43. 1:39see the results coming in here. Uh, the
  44. 1:42red cells are where the AI got it wrong.
  45. 1:45And I can look at these to get a sense
  46. 1:46of where I need to improve things. After
  47. 1:49I've made my changes, I can try again
  48. 1:51and I can see what's changed. Once my
  49. 1:54data sets once my data set starts
  50. 1:57getting bigger, I'll need a way of
  51. 1:58measuring performance, not just
  52. 2:00eyeballing it. And that's where metrics
  53. 2:03come in. Metrics are essentially a score
  54. 2:06that measures the quality of the
  55. 2:07workflow. Metrics can be code-based or
  56. 2:09can use AI to judge performance. So here
  57. 2:13I've got a very simple metric. I am just
  58. 2:15comparing the expected priority for my
  59. 2:17Google sheet with the actual priority
  60. 2:20that's been output by the workflow and
  61. 2:23doing the same for the category. But you
  62. 2:25can also calculate lots of metrics like
  63. 2:27correctness or document rag relevance
  64. 2:29with this feature. I can use the
  65. 2:32evaluations tab to run my evaluation.
  66. 2:36And once it's finished, I can see a roll
  67. 2:38up of my metrics over all my inputs as
  68. 2:42well as results for individual rows. I
  69. 2:45can also see how those metrics have
  70. 2:47evolved over time as I have worked on my
  71. 2:50workflow. So in summary, you should use
  72. 2:53evaluation when building your workflow
  73. 2:55to get an idea of how it performs over a
  74. 2:57handful of different cases, when
  75. 2:59deploying to production to check that
  76. 3:01the edge cases you know about are
  77. 3:03covered, and when making changes to
  78. 3:05check that you haven't accidentally
  79. 3:07broken
  80. 3:08anything. It's a feature we're very
  81. 3:10excited about, and I hope you find it
  82. 3:13useful.

About this transcript

This page contains the full transcript of n8n Evaluation quickstart by n8n, generated from the public captions YouTube serves with the video. The transcript has 576 words across 82 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.