Bjarne Stroupstrup on C++0x

Date: March 18th, 2010 by Author: Daniel Elstner

I’m just back from Aalto University School of Science and Technology in Espoo, Finland, where I attended Bjarne Stroustrup’s roadshow talk on C++0x. In the talk, Stroustrup provided a very nice overview of the features in the C++ standard and how they fit into the evolution of the C++ programming language.

Bjarne Stroustrup made the impression of a down-to-earth guy with a very healthy attitude about C++ features and their development. I was excited to learn that the C++0x draft standard had just been declared final the week before. That means the content of the standard is de-facto finished now, and it will be formalized as an ISO standard in about one year. There sure are quite a number of nice goodies in C++0x, I can’t wait to get my hands dirty on code using it.

For the C++ groupies among us, I took photographs of all presentation slides and of course also of Bjarne Stroustrup himself.

Why I hate Qt

Date: February 25th, 2010 by Author: Daniel Elstner

Although I’m a GNOME and gtkmm guy, my paid work involves Qt. It’s no secret that I have a very strong antipathy towards the Qt mania to assimilate everything into their framework, the culture of their community, and their nonchalant abuse of the C++ programming language. Often, people suggest that I’m exaggerating the faults of Qt, and that there are much worse alternatives out there. Fair enough.

But today I happened onto another one of those little inanities which so often add up to a huge stinking pile. I decided to share this particular example, as I think it may help people to understand why I’m feeling so strongly about Qt.

A while ago, I filed Qt bug #5732, together with a patch to fix the problem. In addition to the fix for the central problem reported in the bug, I also provided a second patch to correct another problem I stumbled upon while I worked on the actual bug fix. The bug was acknowledged by a Qt developer and the code got fixed. They decided to fix it their own way instead of applying my patch, which is fair enough. I may not have agreed with the way it was implemented, but the problem I reported was fixed.

Today, I again looked at the Qt OpenGL code, and happened to stumble upon the OpenGL extension checks. Apparently, the extension checking had now finally been factored out into a separate class, with a more efficient implementation to avoid the numerous temporary memory allocations of the string splitting method that was used before. Of course I was curious and had a look at the code which implements the string parsing.

Here is how it looks like:

// This class can be used to match GL extensions without doing any mallocs. The
// class assumes that the GL extension string ends with a space character,
// which it should do on all conformant platforms. Create the object and pass
// in a pointer to the extension string, then call match() on each extension
// that should be matched. The match() function takes the extension name
// *without* the terminating space character as input.
 
class QGLExtensionMatcher
{
public:
    QGLExtensionMatcher(const char *str)
        : gl_extensions(str), gl_extensions_length(qstrlen(str))
    {}
 
    bool match(const char *str) {
        int str_length = qstrlen(str);
        const char *extensions = gl_extensions;
        int extensions_length = gl_extensions_length;
 
        while (1) {
            // the total length that needs to be matched is the str_length +
            // the space character that terminates the extension name
            if (extensions_length < str_length + 1)
                return false;
            if (qstrncmp(extensions, str, str_length) == 0 && extensions[str_length] == ' ')
                return true;
 
            int split_pos = 0;
            while (split_pos < extensions_length && extensions[split_pos] != ' ')
                ++split_pos;
            ++split_pos; // added for the terminating space character
            extensions += split_pos;
            extensions_length -= split_pos;
        }
        return false;
    }
 
private:
    const char *gl_extensions;
    int gl_extensions_length;
};

According to the log message the piece of code just shown went through internal code review. For comparison, here is the equivalent piece of code from my original patch, which was not considered good enough to be applied:

static bool parse_extensions_string(const char* extensions, const char* name)
{
  const size_t name_length = (name) ? strlen(name) : 0;
 
  Q_ASSERT(name_length > 0);
 
  if (extensions) {
    const char* pos = extensions;
 
    while (const char *const space = strchr(pos, ' ')) {
      const size_t length = space - pos;
 
      if (length == name_length && memcmp(pos, name, length) == 0)
        return true;
 
      pos = space + 1;
    }
    return (strcmp(pos, name) == 0);
  }
  return false;
}

So, instead of just taking the code handed to them on a silver platter, the Qt developers decided to roll their own version. Which I wouldn’t even consider a problem if it weren’t for the fact that the code that went in is longer, less readable, and generally ugly. It’s also broken, because there is not a single word in the OpenGL specification about glGetString() always returning a list terminated by a space character, contrary to what the code comment claims about “all conformant platforms”.

What the fuck is this shit?

Using config.site to set default options

Date: September 21st, 2009 by Author: Daniel Elstner

Just a quick note about a little-used feature of GNU Autoconf: site defaults.

Until recently, I used to have a long list of assignments to module_autogenargs['...'] in my ~/.jhbuildrc file to set up default configure options for various modules. This became unwieldy and there was always one module I forgot.

Autoconf comes to the rescue with its support for site defaults. For illustration, here is my ${prefix}/etc/config.site file:

# Autoconf site defaults
test -n "$enable_silent_rules" || enable_silent_rules=yes
test -n "$enable_static" || enable_static=no
test -n "$enable_warnings" || enable_warnings=fatal

The effect is as if I had passed --enable-silent-rules --disable-static --enable-warnings=fatal to every module’s ./autogen.sh, except that I don’t get any annoying warnings if a module doesn’t actually support a particular option. The test conditionals ensure that command-line arguments have precedence over the default values in the config.site file.

Read the manual for the gory details, if you want to. Otherwise, enjoy!

C++ bindings now in GNOME Documentation Library

Date: September 17th, 2009 by Author: Daniel Elstner

After more than a month of infrastructure work, most of the GNOME C++ binding documentation has now been migrated to the GNOME Documentation Library. This includes the Programming with gtkmm guide and the Doxygen API reference documentation of the core C++ binding modules. Compared to the old documentation hosted on the gtkmm website, we’ve now got a bigger pipe, more bling, and less work for us!

All this would not have been possible without the help of Frédéric Péters, who did the actual integration work in the library-web module and has been very responsive to our numerous questions and tweak requests.

Leading up to the migration, the build infrastructure of the GNOME C++ bindings was completely restructured. All the copy’n'paste of build files had degenerated into an unmaintainable mess. When it became clear that we lacked the infrastructure to properly support cross-references from one documentation module to another, I began to cook up something new to replace the mess. The result of this work is the new mm-common module, which provides the shared build support files for the C++ binding modules. As a novelty, it even comes with documentation!

It is quite a bit of work to convert an existing module to use mm-common, and the transition took more time than I had expected. Despite that, the feedback was very positive, and a number of module maintainers started to convert their modules to mm-common on their own. Many thanks to David King, Fabien Parent, Jonathon Jongsma, José Alburquerque, Krzesimir Nowak and Murray Cumming for your work and feedback!

Non-Recursive Automake Performance

Date: July 30th, 2009 by Author: Daniel Elstner

A few days ago, Murray blogged about Glom’s switch to non-recursive Automake. The topic attracted more feedback than I had expected, and I have been asked if I could give a more detailed account of the speed improvements mentioned by Murray.

For the background, I highly recommend to read the original paper by Peter Miller, Recursive Make Considered Harmful. Here’s an appetizer:

For large UNIX projects, the traditional method of building the project is to use recursive make. On some projects, this results in build times which are unacceptably large, when all you want to do is change one file. In examining the source of the overly long build times, it became evident that a number of apparently unrelated problems combine to produce the delay, but on analysis all have the same root cause.

First off, here are the results for the Glom build, measured before and after the switchover:

make -j2 (clean tree)
337.0 seconds
recursive make
274.4 seconds
non-recursive make
make -j2 (nothing to be done)
4.7 seconds
recursive make
3.5 seconds
non-recursive make
make install
9.6 seconds
recursive make
7.0 seconds
non-recursive make
make clean
3.3 seconds
recursive make
2.9 seconds
non-recursive make

These results were obtained on a Lenovo desktop PC with an Intel Core 2 Duo E7200 at 2.53 GHz and 3 GiB RAM. The operating system is Ubuntu 9.04 (jaunty) for AMD64, installed on an ext4 volume of an encrypted LVM partition. At the time of the measurements, the full GNOME desktop interface was up and running, but otherwise idle. The entire measurement was repeated once with nearly identical results, and the timings presented here are the averages of the two runs.

As can be seen, the switch to non-recursive make reduced the time of a full Glom build by about 19 % on this machine, or more than one minute, which I think is quite decent. However, before you start to convert your own module, be aware that the performance gain you’ll see depends heavily on the specifics of your project. It’s practically guaranteed that you’ll see some improvement, but it could be a meager 0.1 % speedup. However, 40 % are just as possible. To help people gauge the possible effect on their project, I’ll try to outline the factors which I found to matter most in my experience.

Essentially, performance is improved because of two things: First, the time for make to get going and resolve dependencies includes a constant part, which introduces a significant delay even if there is nothing to rebuild. The second problem is that even with parallel make, subdirectories are normally entered in sequence, in order to prevent the number of parallel processes from increasing exponentially. A subdirectory cannot be left until all its targets have finished building. Unless the execution times of all parallel build processes overlap perfectly, one or more processor cores will idle. This effect will be more pronounced if the average number of source files per subdirectory is small. In the pathological case of one file per directory, there will be no parallelism at all. A classic example is the directory hierarchy of source code examples found in many library modules.

The programming language used also plays a role. On average, the idle periods due to imperfect overlap of parallel compilation processes will be longer if compilation is slow. Typically, a C++ source file will require substantially more time to compile than a C source file. The same holds for large source files versus small source files. The optimization level and other compiler settings also influence the compile time per object. With non-recursive make, the stalls due to imperfect overlap will only occur for targets which depend on multiple intermediate targets, such as when linking object files into a library. Even when make needs to wait for a dependency before it can continue with a target, it will still continue building any independent targets. Thus, the builds of multiple independent executables or libraries overlap, thereby hiding the stalls.

With all that mind, I think the 19 % speedup in the case of Glom should be considered above average. The directory hierarchy is deeply nested, with a small number of source files per subdirectory, although not as small as to be pathological. One important factor is that Glom used to build an intermediate static library in each subdirectory, which were later linked to produce the actual target binary. The new non-recursive build system omits these intermediate libraries and simply links all object files from all subdirectories in one go to produce the final target binary.

Automake has included support for non-recursive make for quite a while now, and it turns out to be surprisingly straightforward and easy to use. All you need to get going is AM_PROG_CC_C_O in your configure.ac for C compiler support, and AUTOMAKE_OPTIONS = subdir-objects in your Makefile.am. To refer to files in a subdirectory, just include the relative directory prefix. Automake variable names also include the directory prefix of the file names they are derived from. The *.am files in the Glom repository may be helpful as real-world examples.

Happy make -j∞ !

« Previous Entries