Evan Martin (evan) wrote in evan_tech,
Evan Martin
evan
evan_tech

python's http library / syscalls

In strace'ing some Python code that fetches a web page:
7139  read(5, "H", 1)                   = 1
7139  read(5, "T", 1)                   = 1
7139  read(5, "T", 1)                   = 1
7139  read(5, "P", 1)                   = 1
7139  read(5, "/", 1)                   = 1
7139  read(5, "1", 1)                   = 1
7139  read(5, ".", 1)                   = 1
7139  read(5, "1", 1)                   = 1
7139  read(5, " ", 1)                   = 1
7139  read(5, "3", 1)                   = 1
7139  read(5, "0", 1)                   = 1
7139  read(5, "2", 1)                   = 1
7139  read(5, " ", 1)                   = 1
7139  read(5, "F", 1)                   = 1
7139  read(5, "o", 1)                   = 1
7139  read(5, "u", 1)                   = 1
One syscall per byte.

My Python memcached bindings did the same thing for memcached responses, but nobody's complained yet. I figured if nobody had complained about the behavior of the HTTP library yet, nobody would notice that either...

Perhaps it isn't really a problem? I don't really know what the overhead of syscalls is, but it seems like it shouldn't be insignificant, especially since it's multiplied by the number of bytes in the response. But maybe it is, when compared to the overhead of these interpreted languages? Googling implies that Linux syscalls are fast (at least as compared to FreeBSD's), but I don't know how fast.

What's more likely is that people haven't written performance intensive apps with it yet.


PS: You can compare against Perl with something simple like
strace perl -MLWP::Simple -e "get 'http://www.google.com'"
and see that it uses
read(3, "HTTP/1.0 200 OK\r\nCache-Control: "..., 8192) = 2436
i.e., it reads the entire page in one syscall.
And Ruby's
strace ruby -rnet/http -e 'Net::HTTP.new("www.google.com").get("/")'
at least does 1kb.
Subscribe

  • your vcs sucks

    I've been hacking on some Haskell stuff lately that's all managed in darcs and it's reminded me of an observation I made over two years ago now (see…

  • perl people, explain your language to me

    Every time I use perl I feel mildly positive about it right up until I encounter CPAN. I've never managed to make CPAN work, despite the multitude of…

  • dns attack of doom

    If I've learned anything from the new Kaminsky DNS attack, it's that if you want to keep something a secret while disclosing to a trusted subset of…

  • Post a new comment

    Error

    default userpic
    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.
  • 19 comments

  • your vcs sucks

    I've been hacking on some Haskell stuff lately that's all managed in darcs and it's reminded me of an observation I made over two years ago now (see…

  • perl people, explain your language to me

    Every time I use perl I feel mildly positive about it right up until I encounter CPAN. I've never managed to make CPAN work, despite the multitude of…

  • dns attack of doom

    If I've learned anything from the new Kaminsky DNS attack, it's that if you want to keep something a secret while disclosing to a trusted subset of…