Evan Martin (evan) wrote in evan_tech,
Evan Martin

Wikipedia: I broke down and wrote it in C++. Night and day performance difference. Sigh. (See also: graydon's comment on my last post.)

It turns out at the C level there are two interfaces for fetching MySQL results: one pulls the entire response into the client, and then satisfies repeated requests for rows pull from there, while the other pulls a row at a time. The first requires more memory, but the second locks up the database if the application is slow. The O'Caml (and I imagine other) bindings use the first. No wonder my SELECT cur_title FROM cur was slow.

I also realized pushing the links back into the same database was slow, so now I have a packed binary on-disk representation: 45megabytes, of the form { src: int, count: short, [ { dst: int, weight: short } ] }.

I think my first project will be shortest paths, but that will probably have to wait for another weekend. (I notice that the A* algorithm page links to a thorough and generally great page by Amit, a coworker. (And, now that I poke around, I see his PhD work was on an OO extension for ML!))

  • münchen

    On that note: I'm living in Munich for the next week plus a few days. Do I know anyone around here? (PS: The LJ → PubSubHubbub → Reader…

  • deb/rpm diffing tools

    Dear Linux hackers, Chrome tends to push minor updates (often security) pretty frequently. We'd like to operate as a good member of the Linux…

  • emacs

    I've been using vim for a very long time -- over ten years -- but over those years I've envied more and more the way emacs integrates other software.…

  • Post a new comment


    default userpic
    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.