Brendan Gregg
Brendan Gregg
  • 10
  • 401 683
Velocity 2017: Performance Analysis Superpowers with Linux eBPF
Talk for Velocity 2017 by Brendan Gregg. Abstract: "Advanced performance observability and debugging have arrived built into the Linux 4.x series, thanks to enhancements to Berkeley Packet Filter (BPF, or eBPF) and the repurposing of its sandboxed virtual machine to provide programmatic capabilities to system tracing. Netflix has been investigating its use for new observability tools, monitoring, security uses, and more. This talk will investigate this new technology, which sooner or later will be available to everyone who uses Linux. The talk will dive deep on these new tracing, observability, and debugging capabilities. Whether you’re doing analysis over an ssh session, or via a monitoring GUI, BPF can be used to provide an efficient, custom, and deep level of detail into system and application performance.
This talk will also demonstrate the new open source tools that have been developed, which make use of kernel- and user-level dynamic tracing (kprobes and uprobes), and kernel- and user-level static tracing (tracepoints). These tools provide new insights for file system and storage performance, CPU scheduler performance, TCP performance, and a whole lot more. This is a major turning point for Linux systems engineering, as custom advanced performance instrumentation can be used safely in production environments, powering a new generation of tools and visualizations."
Переглядів: 33 764

Відео

Total Solar Eclipse Real Time of Mt Jefferson, Madras OR, 2017 (2 of 2)
Переглядів 2,2 тис.6 років тому
Filmed from the center line north east of Madras. Video 2 of 2. 1st video is a time-lapse version: ua-cam.com/video/TLqN-2_fnl4/v-deo.html
Total Solar Eclipse Time Lapse of Mt Jefferson, Madras OR, 2017 (1 of 2)
Переглядів 2,9 тис.6 років тому
Time lapse where 1 second equals 60 real seconds. Filmed from the center line north east of Madras. Video 1 of 2. 2nd video is the full 9 minutes in real time: ua-cam.com/video/kHSdxQ6a_3U/v-deo.html
Linux 4.x Tracing: Performance Analysis with bcc/BPF (eBPF)
Переглядів 11 тис.7 років тому
Talk for SCALE15x (2017) by Brendan Gregg. "BPF (Berkeley Packet Filter) has been enhanced in the Linux 4.x series and now powers a large collection of performance analysis and observability tools ready for you to use, included in the bcc (BPF Complier Collection) open source project. BPF nowadays (aka eBPF) can do system tracing, software defined networks, and kernel fast path: much more than ...
Give me 15 minutes and I'll change your view of Linux tracing
Переглядів 106 тис.7 років тому
Demo from the USENIX/LISA 2016 talk: Linux 4.X Tracing Tools: Using BPF Superpowers. Full talk slides and official video will be available at www.usenix.org/conference/lisa16/conference-program/presentation/linux-4x-tracing-tools-using-bpf-superpowers . This video is the 15 (more like 18) minute demo that spans ftrace, perf, and BPF. LISA is a registered trademark of the USENIX Association and ...
SCALE14x Broken Linux Performance Tools (2016)
Переглядів 4 тис.8 років тому
Slides: www.slideshare.net/brendangregg/broken-linux-performance-tools-2016 Talk for SCALE14x (2016). "Broken benchmarks, misleading metrics, and terrible tools. This talk will help you navigate the treacherous waters of Linux performance tools, touring common problems with system tools, metrics, statistics, visualizations, measurement overhead, and benchmarks. You might discover that tools you...
Linux Performance Analysis in 60 seconds
Переглядів 33 тис.8 років тому
See techblog.netflix.com/2015/11/linux-performance-analysis-in-60s.html for more details
Java Mixed-Mode Flame Graphs
Переглядів 4,8 тис.8 років тому
Java Mixed-Mode Flame Graphs
Linux Performance Tools, Brendan Gregg, part 2 of 2
Переглядів 49 тис.8 років тому
Tutorial by Brendan Gregg of Netflix for O'Reilly Velocity conference 2015 Santa Clara. Part 2 of 2. Slides: www.slideshare.net/brendangregg/velocity-2015-linux-perf-tools or velocityconf.com/devops-web-performance-2015/public/schedule/detail/42513 Description: " There are many performance tools nowadays for Linux, but how do they all fit together, and when do we use them? This tutorial explain...
Linux Performance Tools, Brendan Gregg, part 1 of 2
Переглядів 154 тис.8 років тому
Tutorial by Brendan Gregg of Netflix for O'Reilly Velocity conference 2015 Santa Clara. Part 1 of 2. Slides: www.slideshare.net/brendangregg/velocity-2015-linux-perf-tools or velocityconf.com/devops-web-performance-2015/public/schedule/detail/42513 Description: " There are many performance tools nowadays for Linux, but how do they all fit together, and when do we use them? This tutorial explain...

КОМЕНТАРІ

  • @2022spacekw
    @2022spacekw 8 днів тому

    nice!

  • @sealovers1374
    @sealovers1374 15 днів тому

    they present linux but all of them use MacOS

  • @abdullahadel1141
    @abdullahadel1141 2 місяці тому

    Thank you for a wonderful session, much appreciated. 👏

  • @shayantikabhattacharya1985
    @shayantikabhattacharya1985 4 місяці тому

    am I the only one enjoying the sound effect? also how to get the keyboard amd scroll sound effect?

  • @sigridopps3049
    @sigridopps3049 6 місяців тому

    unprepared and confused

    • @ManiBalajiC
      @ManiBalajiC Місяць тому

      thats why he is the head of the netflix system performance.

  • @brewtaldeafmedal1262
    @brewtaldeafmedal1262 6 місяців тому

    next level hand movement

  • @elabeddhahbi3301
    @elabeddhahbi3301 6 місяців тому

    I'm speechless than you

  • @iancu_de_hunedoara
    @iancu_de_hunedoara 7 місяців тому

    the sounds are extremely annoying. i was excited to watch the video but it is too much spoiled

  • @diegoporras7769
    @diegoporras7769 9 місяців тому

    Amazing talk! Lots of great info, a little too fast. I had to watch it at x0.75 speed jaja

  • @tomknud
    @tomknud 10 місяців тому

    Is that a bee or a dragon fly? Isn't that CSRA in SHV?

  • @ManishSharma-qs3sv
    @ManishSharma-qs3sv 11 місяців тому

    Hi Sir, Can you post more videos as your videos are very very knowledgeable and I am learning a lot ? Please post more videos

  • @edubmf
    @edubmf Рік тому

    Really good, what an interesting guy Brendan Gregg is.

  • @ytdlgandalf
    @ytdlgandalf Рік тому

    he didn't lie with that title.

  • @ProfessionalBabyEater
    @ProfessionalBabyEater Рік тому

    you should try screaming at babies next

  • @raulsaavedra709
    @raulsaavedra709 Рік тому

    Hi Brendan, reading your excellent Systems Performance book. Looked up the reference in page 20 and found this video, but the link in the book, same as the one in this video's description box, no longer works. Managed to still find the blog post though, but has a different URL starting with netflixtechblog

  • @king0s
    @king0s Рік тому

    This is golden

  • @tezcanaslan2877
    @tezcanaslan2877 Рік тому

    I hope you are continuing to scream at hard drives

  • @ArslanFarooqQureshi
    @ArslanFarooqQureshi Рік тому

    You sir, are a legend.

  • @praveshgaire3437
    @praveshgaire3437 Рік тому

    Would have been better if video was of 60 secs 😄

  • @MaxCoplan
    @MaxCoplan Рік тому

    Great talk. Can’t imagine how annoying the sound effects were for the in-person audience 😂

  • @user-nc7ui5iu5s
    @user-nc7ui5iu5s Рік тому

    666

  • @rchaudry
    @rchaudry 2 роки тому

    Thank you!

  • @solomontan1524
    @solomontan1524 2 роки тому

    This is something Id like to see in my UA-cam shorts recommendation

  • @acidixzkinght9575
    @acidixzkinght9575 2 роки тому

    Best lecture I have ever seen

  • @bimalacharya9020
    @bimalacharya9020 2 роки тому

    fantastic !

  • @johannesboneschanscher4264
    @johannesboneschanscher4264 2 роки тому

    Although it has been a couple of years since this presentation a lot of pitfalls are still relevant today. I cannot help to point out that the tcpdump for a 10Mbit link very likely dropped packets because tcpdump was run without specifying -n option, making tcpdump do dns lookups to resolve addresses to hostnames.

  • @robsonmagno5478
    @robsonmagno5478 2 роки тому

    what is the advantage and disadvantage of java/jvm on freebsd compared to Linux? do you have any performance advantage?

  • @QuadDerrick
    @QuadDerrick 3 роки тому

    "more than 80 characters is a sin in unix world" :):):) didnt know but yea, makes sence. Norway

  • @lubacarolinanarvaezerazo6964
    @lubacarolinanarvaezerazo6964 3 роки тому

    like.

  • @supertren
    @supertren 3 роки тому

    "My name is Brendan"....he is also modest.

  • @tungvu4339
    @tungvu4339 3 роки тому

    maybe this one is a little bit advanced

  • @davidleal714
    @davidleal714 3 роки тому

    Thank you man, I will definitely copy yours methods

  • @AdolfoNeto
    @AdolfoNeto 3 роки тому

    1:38 and 15:27 perf_events Read about it here too www.brendangregg.com/perf.html

  • @karthiknedunchezhiyan1171
    @karthiknedunchezhiyan1171 3 роки тому

    Very informative! keep going!

  • @pixelPlex
    @pixelPlex 3 роки тому

    Which tools are used for monitoring network usage/performance of a process?

  • @yunuskoning7584
    @yunuskoning7584 3 роки тому

    Hi Brendan, You are not by any chance available for freelance performance trouble shooting projects?

  • @GisleVanem00
    @GisleVanem00 3 роки тому

    I see Linux is still *built by nerds and for the nerds*.

  • @azufendusgarendum6583
    @azufendusgarendum6583 3 роки тому

    Thank you very much

  • @retropaganda8442
    @retropaganda8442 4 роки тому

    Why would you need unicode to make a grey scale in your terminal ?

  • @retropaganda8442
    @retropaganda8442 4 роки тому

    wtf is this macintosh :(

  • @retropaganda8442
    @retropaganda8442 4 роки тому

    wtf is this macintosh :(

  • @adrian_88
    @adrian_88 4 роки тому

    Actually, looking in "man syscall" for the calling convention is slightly wrong, because that manpage describes the calling convention for doing a system call, but you're tracing just a regular function call. Fortunately, the convention is mostly the same, except for the 4th argument, which for a function call is rcx, but the manpage would have told you that it's r10.

  • @charle09111
    @charle09111 4 роки тому

    OMG!! I really need it.

  • @ankitsrivastava6066
    @ankitsrivastava6066 4 роки тому

    FIO - 09:55

  • @harshbhadoria7200
    @harshbhadoria7200 4 роки тому

    Hi guys, can anyone tell how is it possible to collect all metrics from Linux and visualize them in windows OS? How to make a bridge or connection between Linux and Windows and what are the technologies to be used.

  • @subhabera5775
    @subhabera5775 4 роки тому

    Thanks Brendan :) You are awesome.

  • @bhanureddy8780
    @bhanureddy8780 4 роки тому

    This is amazing. true love for the bits and bytes

  • @domaincontroller
    @domaincontroller 4 роки тому

    05:03 tcplife, tcpdump 08:19 linux 4.4+ 19:40 linux perf analysis in 60s

  • @domaincontroller
    @domaincontroller 4 роки тому

    00:11 ss 00:26 iptraf 00:37 iotop 00:54 slabtop 01:17 pcstat 01:38 perf_events, part of the linux source code 02:02 tiptop, PMC's 03:38 rdmsr ****************************** Advanced tracers ****************************** 05:28 linux observability tools 09:50 imbench, benchmarking 10:40 pchar *************************** 11:18 tuning tools ************************** Static tools 13:07 ? storage devices 14:02 routing table, netstat, ip route get 15:31 perf_events, great cpu profiler, perf report 21:00 workflow for perf_events 23:30 linux tracing tools 24:17 choosing a tracer 25:01 ftrace in the linux source code,

  • @domaincontroller
    @domaincontroller 4 роки тому

    07:08 system is running slow, commands 07:11 top (10) 07:31 iotop 07:45 iostat -xz 1 (6) 07:57 netstat 08:14 ss 08:34 dstat 08:43 sar -n DEV 1 (8) 08:59 vmstat (3) 09:10 | 09:49 strace 10:17 what do they mean by the system is slow, latency 11:51 packages, sysstat, procps, coreutils ============================================ 12:07 Anti-methodologies ============================================ 12:13 street light anti-method 13:25 drunk man anti-method 13:59 blame someone else anti-method 14:43 | 16:02 problem statement method ============================================ 16:30 workload characterization method 17:48 USE method, utilization, saturation, errors 18:58 USE method for hardware systems 21:20 off-CPU Analysis 22:36 CPU profile method 24:26 RTFM Method 24:54 reading linux source code, jvm 26:50 How do you measure them ###################### 27:07 observability tools ###################### 27:18 uptime (1) 30:02 top (or htop) 31:57 ps -ef 32:28 vmstat 1 (3) 33:28 instant-xmdz 36:02 mpstat -P ALL 1 (4) 36:14 free -m (7) 36:46 latency is much higher, can you debug it 42:32 observability tools: basic *********************** intermediate *********************** 42:48 strace 44:21 tcpdump 46:15 net stat, nicstat 46:49 pidstat 1 (5) 47:21 swapon 47:33 lsof 47:58 sar (8) 50:38 app is taking forever 50:45 | 51:17 pidstat, system time 50:57 system time, iostat 51:26 am i swapping ? vmstat 51:51 strace