[Warning:
long post. Short version: Building in SharpDevelop goes multi-threaded,
scroll down for benchmark results]
SharpDevelop
3.0.0.2694 contains a rewritten build system. It
has some new features that might be interesting for AddIn developers, but one new
feature is directly useful for users, or more specifically, users with multi-core
CPUs.
Here
a quick history on build systems in SharpDevelop:
SharpDevelop
2.0 (the first version to use MSBuild), compiled solutions by telling MSBuild to build the
.sln file.
SharpDevelop 2.1 and 2.2 compile solutions by telling MSBuild to build the projects
individually.
When
writing the build engine for SharpDevelop 2.1, I tried to make it multi-threaded so
that two projects that are not dependent on each other can be built in parallel. However,
this did not work: MSBuild makes use Environment.CurrentDirectory; and if two MSBuild
instances run in the same process at the same time, they overwrite the CurrentDirectory
and cause the other instance to search files in the wrong location.
Now
I solved this problem by moving the build into worker processes. However, this means
that all MSBuild events are serialized and remoted to the host process, where
SharpDevelop AddIns need to have access to them - e.g. the CodeAnalysis AddIn registers itself
as MSBuild logger, waits for the TaskStarted, ErrorRaised, WarningRaised and TaskFinished
events and fixes certain FxCop error messages (some FxCop errors are not associated
with a line number, but with a class/member name). Other SharpDevelop AddIns might
decide that certain MSBuild events should be displayed in some form, e.g. in the output
window. The only way to give AddIns access to all MSBuild events like they had before
is to serialize all events and send them to the SharpDevelop process. When doing a
build where there's relatively few work but lots of events (e.g. "Clean solution"),
the worker is sending up to 1 MB/s of serialized events to the
host.
This
had a noticeable impact on performance, so I've kept the first worker in the SharpDevelop
process, and only additional workers get their own process. So if you
use only one worker, you won't see any worker process and should get the same build
performance as SharpDevelop 2.x.
So
even though building a project has multiple parts, of which some are waiting for the
disk and others are waiting for the CPU; the overhead of using a worker process will
cause the build to get slower. (MSBuild has to check if input files changed, start
the compiler; the compiler will load input files, actually compile, write the output
assembly; and some other stuff happens in the obj\ directory that you probably don't
want to hear about)
But
if you have a dual core processor, the CPU overhead doesn't hurt: a little overhead
for one core is better than not using that core at all (this is assuming the
compiler you use is not multithreaded).
But
enough talking, let's get some benchmark results:
I
used SharpDevelop 3.0.0.2694 to compile the source code for SharpDevelop 2.2.1:
AMD 3500+ (single core), 2 GB RAM, Vista Ultimate (32 bit), Areo enabled:
Rebuild:
One worker: 34.3, 34.1, 34.6, 34.7, 34.3 : Average 34.4 seconds
Two workers: 39.6, 38.7, 38.1, 37.0, 36.6 : Average 38.0 seconds
Build
(no changes, so it's not compiling, but just checking everything for changes):
One worker: 8.5, 8.0, 8.3, 8.0, 8.0 : Average
8.2 seconds
Two workers: 13.4, 13.9, 14.0, 12.8, 13.1 : Average 13.4 seconds
The
number of MSBuild events (Target started, Task started, Log Message...) is not much
different between Rebuild and Build (no changes), so we get a constant overhead of
about 5 seconds for the SharpDevelop solution on my desktop computer with a single
core processor.
Here
finally is the interesting part: Test results from my dual core notebook:
Core
Duo T2300, 1 GB RAM, Vista Home Premium (32 bit), Aero disabled:
Rebuild:
One worker: 35.2, 34.9, 35.2, 35.0, 34.3 : Average 35.1 seconds
Two workers: 25.6, 23.9, 24.2, 23.5, 23.8 : Average 24.2 seconds
Build
(no changes):
One worker: 7.4, 7.1, 7.2, 7.8, 7.1 : Average 7.3 seconds
Two workers: 6.9, 6.8, 6.7, 6.6, 6.2 : Average 6.6 seconds
For build
(no changes), the overhead plays a big role, but using two workers still made the
build a little faster.
But
for the real job when there's something to compile, using two workers really rocks:
My build is now 30% faster.
SharpDevelop
3.0 allows you to set the number of workers under Tools > Options > Project
and solution > Number of projects to build in parallel. The default value for single-core
processors is 1, for other processors it's 2. I don't know if going above 2 will have
benefits for quad-core users as the overhead (both CPU for sending events to the host
and memory because of the additional processes) increases with the number of workers.
Please
leave a comment here with your test results - I'm looking for feedback from other
multi-core users.
And
if you're doing your own benchmark, some notes on how I did my benchmark:
- make sure enough RAM is free so that not only the system does not swap, but Windows
has enough room to keep the files you are compiling in memory (this is generally a
good idea if you want your system to be fast)
- don't count the first test run as the files are not yet in memory there
- reuse the same SharpDevelop instance for all test runs
- if you leave less than 60 seconds between test runs, SharpDevelop will re-use the
worker process, otherwise it is shut down and has to be restarted. I did my tests
re-using a single worker process. It might be interesting to see if the overhead
of restarting the worker process is worth keeping it longer than 60 seconds.
