Concerto, a cross-platform .net profiler

November 12, 2010

I'm going to introduce the basics behind Concerto, the cross-platform profiler we've developed to help us optimizing Plastic SCM.

The problem


The problem we face is simple: we need to optimize our code in Mono/Linux, Mono/Solaris, Mono/Mac and .NET/Windows. The best way to find issues is to run exactly the same test on the same hardware but with different OS (ok, Mac and Solaris are slightly more complex, right?, but for the main ones, Linux/Windows, the same iron will work), and then check the results.

We've been doing that for months but we always missed good comparable profiling data. You know, we do use things like AQTime on Windows and then the Mono Profiler on Linux but the generated data is not easy to compare.

We needed a different tool.

Let's start the music


We had the following idea: let's instrument the code to add some instructions to measure the time spent on each method, this way we can use the same instrumented assemblies to run with .NET and Mono and then we can compare.

Cecil was there to help doing the tough instrumentation part, so I emailed our resident Mono hacker: Dick Porter, and told him the idea. It took him a few hours to come up with a prototype, and has been refining it for the last week or so...

And, since it was all about instrumenting code... Dick named it Concerto.

Welcome Concerto


Here is how the toy works:


$ mono Instrument.exe --help
Usage: Instrument [OPTIONS] assemblies
Instrument one or more assemblies to
record profiling data.
If only one assembly is specified,
instrumented assembly output name and
data filename can be set. Otherwise
defaults are chosen for each assembly.

Options:
-v, --verbose Increase verbosity
-h, -?, --help Show this message and exit
-f, --filename=VALUE The filename where profiling data is written to
at runtime
-o, --out=VALUE The filename where the instrumented assembly is
written
-p, --private Include private types
-c, --class=VALUE The specific class to instrument (can be given
more than once)



Usage is pretty simple:

1) run Instrument.exe on an assembly of your choice
2) copy the output assembly back to your application, along with the generated Concerto-blah.dll
3) run your application
4) Look at the output file with mprof-decoder (or equivalent)

For example:


$ ls -l hello.* Concerto*dll *.mprof
-rwxr-xr-x. 1 dick dick 13312 2010-11-08 14:06 Concerto.dll
-rw-r--r--. 1 dick dick 384 2010-11-04 20:10 hello.cs
-rwxr-xr-x. 1 dick dick 3072 2010-11-05 16:51 hello.exe

$ mono Instrument.exe -vvv hello.exe
Instrumenting hello.exe, creating hello.exe.ins
The helper assembly is Concerto-hello_exe.dll
Data shall be written to concerto-hello_exe.mprof
Instrumenting class hello
Instrumenting method System.Void hello::.ctor()
Instrumenting method System.Int32 hello::DoStuff(System.Int32)
Instrumenting method System.Void hello::DoMoreStuff()
Instrumenting method System.Void hello::Main()
Done.

$ ls -l hello.* Concerto*dll
-rw-r--r--. 1 dick dick 13824 2010-11-12 11:53 Concerto-hello_exe.dll
-rwxr-xr-x. 1 dick dick 13312 2010-11-08 14:06 Concerto.dll
-rw-r--r--. 1 dick dick 384 2010-11-04 20:10 hello.cs
-rwxr-xr-x. 1 dick dick 3072 2010-11-05 16:51 hello.exe
-rw-r--r--. 1 dick dick 3584 2010-11-12 11:53 hello.exe.ins

$ mono hello.exe.ins
Hello, world!
1
1
2

$ ls -l hello.* Concerto*dll *.mprof
-rw-r--r--. 1 dick dick 13824 2010-11-12 11:53 Concerto-hello_exe.dll
-rwxr-xr-x. 1 dick dick 13312 2010-11-08 14:06 Concerto.dll
-rw-r--r--. 1 dick dick 365 2010-11-12 11:54 concerto-hello_exe.mprof
-rw-r--r--. 1 dick dick 384 2010-11-04 20:10 hello.cs
-rwxr-xr-x. 1 dick dick 3072 2010-11-05 16:51 hello.exe
-rw-r--r--. 1 dick dick 3584 2010-11-12 11:53 hello.exe.ins


Now let's inspect the output with the mprof-decoder


$ mono mprof-decoder.exe concerto-hello_exe.mprof

------------------------------------------------
Reporting execution time (on 4 methods)
97.19% (0.007282s) hello.System.Void hello::Main()
2.48% (0.000186s) hello.System.Int32 hello::DoStuff(System.Int32)
1 calls from hello.System.Void hello::Main()
1 calls from hello.System.Void hello::DoMoreStuff()
0.33% (0.000025s) hello.System.Void hello::DoMoreStuff()
1 calls from hello.System.Void hello::Main()


------------------------------------------------
Reporting execution time by stack frame
97.19% (0.007282s, 1 calls) hello.System.Void hello::Main()
2.33% (0.000169s, 1 calls) hello.System.Int32 hello::DoStuff(System.Int32)
0.34% (0.000025s, 1 calls) hello.System.Void hello::DoMoreStuff()
66.67% (0.000017s, 1 calls) hello.System.Int32 hello::DoStuff(System.Int32)


Finally an example of picking classes to instrument would look like this:

Get the list of classes with verbosity level 2:


$ mono Instrument.exe -vv plastictcpchannel.dll
Instrumenting plastictcpchannel.dll, creating plastictcpchannel.dll.ins
The helper assembly is Concerto-plastictcpchannel_dll.dll
Data shall be written to concerto-plastictcpchannel_dll.mprof
Instrumenting class Codice.Channels.ClientSinkProvider
Instrumenting class Codice.Channels.ClientSink
Instrumenting class Codice.Channels.PlasticBinaryServerFormatterSink
Instrumenting class Codice.Channels.PlasticBinaryServerFormatterSinkProvider
Instrumenting class Codice.Channels.PlasticTcpChannel
Done.


Pick a couple of classes to instrument:


$ mono Instrument.exe -vv -c
Codice.Channels.PlasticBinaryServerFormatterSinkProvider -c
Codice.Channels.ClientSinkProvider plastictcpchannel.dll
Instrumenting plastictcpchannel.dll, creating plastictcpchannel.dll.ins
The helper assembly is Concerto-plastictcpchannel_dll.dll
Data shall be written to concerto-plastictcpchannel_dll.mprof
Instrumenting class Codice.Channels.ClientSinkProvider
Instrumenting class Codice.Channels.PlasticBinaryServerFormatterSinkProvider
Done.


Enjoy!

4 comments :

knocte said...

Awesome! Are you publishing the sources? If yes, which license?

It would be great to have an addin for using this in MonoDevelop!

RichB said...

Does it work with strong named assemblies?

Dick said...

To be honest I hadn't thought about strong named assemblies. The short answer is "unlikely" as we're changing the assembly to add the instrumentation. I'm not sure if Cecil takes out the strong name metadata from the assembly without trying it to see.

On JB's blog when he announced the new version of Cecil that we're using, he says "Better support for pdb and mdb files and strong name assemblies" is a new feature, but doesn't go into details.

I'll add this to my todo list.

- Dick

pablo said...

@knocte: yes, my plan is to publish the sources, maybe next week, and with a clearly open license.

I'd also like to add some GUI (maybe a web based one) on top, and with realtime monitoring, that would be huge.

And I'm eager to test it on our 100 nodes cluster to compare Windows/Linux performance. And then make plastic even faster! :)

Real Time Web Analytics