Skip to content

Performance profiling with pprof

Haotian edited this page Dec 20, 2018 · 1 revision

Justitia supports performance profiling with pprof.

Enable pprof endpoint

Make sure pprof endpoint is enabled, checkout configuration file (e.g. ~/.justitia/justitia.yaml):

monitor:
  ...
  pprof:
    enabled: true
    port: 6666

How to get performace data

While justitia is running, we can get performance data through port http://<peer-ip>:6666/debug/pprof:

  • use the pprof tool to look at the heap profile:

    go tool pprof http://<peer-ip>:6666/debug/pprof/heap
    
  • Or to look at a 30-second CPU profile:

    go tool pprof http://<peer-ip>:6666/debug/pprof/profile?seconds=30
    
  • Or to look at the goroutine blocking profile, after calling runtime.SetBlockProfileRate in your program:

    go tool pprof http://<peer-ip>:6666/debug/pprof/block
    
  • Or to collect a 5-second execution trace:

    wget http://<peer-ip>:6666/debug/pprof/trace?seconds=5
    
  • Or to look at the holders of contended mutexes, after calling runtime.SetMutexProfileFraction in your program:

    go tool pprof http://<peer-ip>:6666/debug/pprof/mutex
    

To view all available profiles, open http://<peer-ip>:6666/debug/pprof/ in your browser.

For a study of the facility in action, visit https://blog.golang.org/2011/06/profiling-go-programs.html.

Take CPU profiling as an example.

Run the following command to generate CPU performance data:

```
go tool pprof http://<peer-ip>:6666/debug/pprof/profile?seconds=30
```

Usually after 30 seconds, this command will lead to interective mode.

Then you can type:

  • text to print out top 10 ,
  • or svg to generate graph file under current folder,
  • or web to directly open svg in web browser,
  • or proto to generate the profile in compressed protobuf format under current folder,
  • or help to see other commands.

Even if you don't use proto command, it will also generate the profile in compressed protobuf formatted file under $HOME/pprof in name of pattern pprof.main.samples.cpu.NNN.pb.gz.

Generate flame graph for a better inspection

We can use uber/go-torch to generate flame graph. And go-torch needs FlameGraph to work, so to install them:

git clone --depth=1 https://github.com/brendangregg/FlameGraph.git ~/.flamegraph
export PATH=$PATH:~/.flamegraph

go get github.com/uber/go-torch

Then use proto formatted data to generate flame graph, for example:

go-torch -b ~/pprof/pprof.main.samples.cpu.001.pb.gz -f cpu.torch.svg

Open generated svg in chrome, and tune away!