evan_tech -- gitweb/perl speed

12:33 pm, 1 Dec 07

gitweb/perl speed

gitweb is nice but slow -- about 400ms to produce the index page for a directory that contains only one project. (Not really worth running mod_perl for, y'know?) By unfair contrast, Google's memory-resident server provides me search results in about 150ms, including HTTP transfer time.

About 200ms of gitweb's runtime seems to be Perl startup time, which I gauged by measuring a simple script that just imports the same things gitweb does and exits immediately. It opens 33 files from directories with "perl" in their name, mostly .pms but a couple .sos as well. Is there a way to "prelink" a perl program?

Perl spends another 140ms around calls to read() on the gitweb script. I guess it's probably parsing at the same time? (The script is 160k(!) of perl code.) I found you can run perlcc -B foo.cgi to produce bytecode, which speeds things up by a bit over 100ms. (Without the -B flag it tries to compile via C and aborts with some internal failure message. The manpage for the package has lots of scary-looking "do not use this for production" messages.) But the module loads are still external, and continue to dominate the startup time.

Anonymous

reply to this
- evan Oh, that does look much nicer!
  
  reply to this

mustikapu	There's a caching mechanism in perl to load compiled .pmc files, but it's not really used. I'd just use FastCGI.
reply to this

wolverian	Er, that was me.
reply to this

evan	That either requires a long-running server (waste of memory, given that my gitweb will likely get one hit a month) or still has the slow boot time that the existing setup does.
reply to this

rjpower

Mercurial has a similar function builtin, but starts it's own internal webserver instead of running as a CGI. It takes a second to startup, but it's snappy once it does. I'm not sure if gitweb has a similar option, or if it would be easy to add it?

OTOH, I tend to use the web interface for brief periods of time when I'm looking at stuff while coding, rather then for serving up content. I'm not sure how appropriate a perl/python webserver is for routinely dishing up content.

reply to this

evan

But in that case, I'd be running a {git,hg}web server all the time when in fact it'll only ever get about 10 hits. I'd rather have it live on disk and just start fast.

I switched it to use cgit, which generates pages too quickly to measure. And it has a neat design where it stuffs everything it generates into an on-disk cache so it only ever has to generate a given page.

http://neugierig.org/software/git/

reply to this

rjpower	Wow, that is quick. Nice. You could also just inflate your ego to the point where you believe a full on separate webserver is necessary just to handle all of the traffic to gitweb :).
reply to this