![Brendan Gregg](/img/default-banner.jpg)
- 10
- 401 683
Brendan Gregg
Приєднався 14 чер 2011
Velocity 2017: Performance Analysis Superpowers with Linux eBPF
Talk for Velocity 2017 by Brendan Gregg. Abstract: "Advanced performance observability and debugging have arrived built into the Linux 4.x series, thanks to enhancements to Berkeley Packet Filter (BPF, or eBPF) and the repurposing of its sandboxed virtual machine to provide programmatic capabilities to system tracing. Netflix has been investigating its use for new observability tools, monitoring, security uses, and more. This talk will investigate this new technology, which sooner or later will be available to everyone who uses Linux. The talk will dive deep on these new tracing, observability, and debugging capabilities. Whether you’re doing analysis over an ssh session, or via a monitoring GUI, BPF can be used to provide an efficient, custom, and deep level of detail into system and application performance.
This talk will also demonstrate the new open source tools that have been developed, which make use of kernel- and user-level dynamic tracing (kprobes and uprobes), and kernel- and user-level static tracing (tracepoints). These tools provide new insights for file system and storage performance, CPU scheduler performance, TCP performance, and a whole lot more. This is a major turning point for Linux systems engineering, as custom advanced performance instrumentation can be used safely in production environments, powering a new generation of tools and visualizations."
This talk will also demonstrate the new open source tools that have been developed, which make use of kernel- and user-level dynamic tracing (kprobes and uprobes), and kernel- and user-level static tracing (tracepoints). These tools provide new insights for file system and storage performance, CPU scheduler performance, TCP performance, and a whole lot more. This is a major turning point for Linux systems engineering, as custom advanced performance instrumentation can be used safely in production environments, powering a new generation of tools and visualizations."
Переглядів: 33 764
Відео
Total Solar Eclipse Real Time of Mt Jefferson, Madras OR, 2017 (2 of 2)
Переглядів 2,2 тис.6 років тому
Filmed from the center line north east of Madras. Video 2 of 2. 1st video is a time-lapse version: ua-cam.com/video/TLqN-2_fnl4/v-deo.html
Total Solar Eclipse Time Lapse of Mt Jefferson, Madras OR, 2017 (1 of 2)
Переглядів 2,9 тис.6 років тому
Time lapse where 1 second equals 60 real seconds. Filmed from the center line north east of Madras. Video 1 of 2. 2nd video is the full 9 minutes in real time: ua-cam.com/video/kHSdxQ6a_3U/v-deo.html
Linux 4.x Tracing: Performance Analysis with bcc/BPF (eBPF)
Переглядів 11 тис.7 років тому
Talk for SCALE15x (2017) by Brendan Gregg. "BPF (Berkeley Packet Filter) has been enhanced in the Linux 4.x series and now powers a large collection of performance analysis and observability tools ready for you to use, included in the bcc (BPF Complier Collection) open source project. BPF nowadays (aka eBPF) can do system tracing, software defined networks, and kernel fast path: much more than ...
Give me 15 minutes and I'll change your view of Linux tracing
Переглядів 106 тис.7 років тому
Demo from the USENIX/LISA 2016 talk: Linux 4.X Tracing Tools: Using BPF Superpowers. Full talk slides and official video will be available at www.usenix.org/conference/lisa16/conference-program/presentation/linux-4x-tracing-tools-using-bpf-superpowers . This video is the 15 (more like 18) minute demo that spans ftrace, perf, and BPF. LISA is a registered trademark of the USENIX Association and ...
SCALE14x Broken Linux Performance Tools (2016)
Переглядів 4 тис.8 років тому
Slides: www.slideshare.net/brendangregg/broken-linux-performance-tools-2016 Talk for SCALE14x (2016). "Broken benchmarks, misleading metrics, and terrible tools. This talk will help you navigate the treacherous waters of Linux performance tools, touring common problems with system tools, metrics, statistics, visualizations, measurement overhead, and benchmarks. You might discover that tools you...
Linux Performance Analysis in 60 seconds
Переглядів 33 тис.8 років тому
See techblog.netflix.com/2015/11/linux-performance-analysis-in-60s.html for more details
Linux Performance Tools, Brendan Gregg, part 2 of 2
Переглядів 49 тис.8 років тому
Tutorial by Brendan Gregg of Netflix for O'Reilly Velocity conference 2015 Santa Clara. Part 2 of 2. Slides: www.slideshare.net/brendangregg/velocity-2015-linux-perf-tools or velocityconf.com/devops-web-performance-2015/public/schedule/detail/42513 Description: " There are many performance tools nowadays for Linux, but how do they all fit together, and when do we use them? This tutorial explain...
Linux Performance Tools, Brendan Gregg, part 1 of 2
Переглядів 154 тис.8 років тому
Tutorial by Brendan Gregg of Netflix for O'Reilly Velocity conference 2015 Santa Clara. Part 1 of 2. Slides: www.slideshare.net/brendangregg/velocity-2015-linux-perf-tools or velocityconf.com/devops-web-performance-2015/public/schedule/detail/42513 Description: " There are many performance tools nowadays for Linux, but how do they all fit together, and when do we use them? This tutorial explain...
nice!
they present linux but all of them use MacOS
Thank you for a wonderful session, much appreciated. 👏
am I the only one enjoying the sound effect? also how to get the keyboard amd scroll sound effect?
unprepared and confused
thats why he is the head of the netflix system performance.
next level hand movement
I'm speechless than you
the sounds are extremely annoying. i was excited to watch the video but it is too much spoiled
Amazing talk! Lots of great info, a little too fast. I had to watch it at x0.75 speed jaja
Is that a bee or a dragon fly? Isn't that CSRA in SHV?
Hi Sir, Can you post more videos as your videos are very very knowledgeable and I am learning a lot ? Please post more videos
Really good, what an interesting guy Brendan Gregg is.
he didn't lie with that title.
you should try screaming at babies next
Hi Brendan, reading your excellent Systems Performance book. Looked up the reference in page 20 and found this video, but the link in the book, same as the one in this video's description box, no longer works. Managed to still find the blog post though, but has a different URL starting with netflixtechblog
This is golden
I hope you are continuing to scream at hard drives
You sir, are a legend.
Would have been better if video was of 60 secs 😄
Great talk. Can’t imagine how annoying the sound effects were for the in-person audience 😂
🤣
666
Thank you!
This is something Id like to see in my UA-cam shorts recommendation
Best lecture I have ever seen
fantastic !
Although it has been a couple of years since this presentation a lot of pitfalls are still relevant today. I cannot help to point out that the tcpdump for a 10Mbit link very likely dropped packets because tcpdump was run without specifying -n option, making tcpdump do dns lookups to resolve addresses to hostnames.
what is the advantage and disadvantage of java/jvm on freebsd compared to Linux? do you have any performance advantage?
"more than 80 characters is a sin in unix world" :):):) didnt know but yea, makes sence. Norway
like.
"My name is Brendan"....he is also modest.
maybe this one is a little bit advanced
Thank you man, I will definitely copy yours methods
1:38 and 15:27 perf_events Read about it here too www.brendangregg.com/perf.html
Very informative! keep going!
Which tools are used for monitoring network usage/performance of a process?
Hi Brendan, You are not by any chance available for freelance performance trouble shooting projects?
I see Linux is still *built by nerds and for the nerds*.
Thank you very much
Why would you need unicode to make a grey scale in your terminal ?
wtf is this macintosh :(
wtf is this macintosh :(
Actually, looking in "man syscall" for the calling convention is slightly wrong, because that manpage describes the calling convention for doing a system call, but you're tracing just a regular function call. Fortunately, the convention is mostly the same, except for the 4th argument, which for a function call is rcx, but the manpage would have told you that it's r10.
OMG!! I really need it.
FIO - 09:55
Hi guys, can anyone tell how is it possible to collect all metrics from Linux and visualize them in windows OS? How to make a bridge or connection between Linux and Windows and what are the technologies to be used.
Thanks Brendan :) You are awesome.
This is amazing. true love for the bits and bytes
05:03 tcplife, tcpdump 08:19 linux 4.4+ 19:40 linux perf analysis in 60s
00:11 ss 00:26 iptraf 00:37 iotop 00:54 slabtop 01:17 pcstat 01:38 perf_events, part of the linux source code 02:02 tiptop, PMC's 03:38 rdmsr ****************************** Advanced tracers ****************************** 05:28 linux observability tools 09:50 imbench, benchmarking 10:40 pchar *************************** 11:18 tuning tools ************************** Static tools 13:07 ? storage devices 14:02 routing table, netstat, ip route get 15:31 perf_events, great cpu profiler, perf report 21:00 workflow for perf_events 23:30 linux tracing tools 24:17 choosing a tracer 25:01 ftrace in the linux source code,
07:08 system is running slow, commands 07:11 top (10) 07:31 iotop 07:45 iostat -xz 1 (6) 07:57 netstat 08:14 ss 08:34 dstat 08:43 sar -n DEV 1 (8) 08:59 vmstat (3) 09:10 | 09:49 strace 10:17 what do they mean by the system is slow, latency 11:51 packages, sysstat, procps, coreutils ============================================ 12:07 Anti-methodologies ============================================ 12:13 street light anti-method 13:25 drunk man anti-method 13:59 blame someone else anti-method 14:43 | 16:02 problem statement method ============================================ 16:30 workload characterization method 17:48 USE method, utilization, saturation, errors 18:58 USE method for hardware systems 21:20 off-CPU Analysis 22:36 CPU profile method 24:26 RTFM Method 24:54 reading linux source code, jvm 26:50 How do you measure them ###################### 27:07 observability tools ###################### 27:18 uptime (1) 30:02 top (or htop) 31:57 ps -ef 32:28 vmstat 1 (3) 33:28 instant-xmdz 36:02 mpstat -P ALL 1 (4) 36:14 free -m (7) 36:46 latency is much higher, can you debug it 42:32 observability tools: basic *********************** intermediate *********************** 42:48 strace 44:21 tcpdump 46:15 net stat, nicstat 46:49 pidstat 1 (5) 47:21 swapon 47:33 lsof 47:58 sar (8) 50:38 app is taking forever 50:45 | 51:17 pidstat, system time 50:57 system time, iostat 51:26 am i swapping ? vmstat 51:51 strace
7:46 iostat -x 1
thx
Thanks for this list!