Wikipedia: I broke down and wrote it in C++. Night and day performance difference. Sigh. (See also: 
graydon's comment on my last post.)
It turns out at the C level there are two interfaces for fetching MySQL results: one pulls the entire response into the client, and then satisfies repeated requests for rows pull from there, while the other pulls a row at a time. The first requires more memory, but the second locks up the database if the application is slow. The O'Caml (and I imagine other) bindings use the first. No wonder my SELECT cur_title FROM cur was slow.
I also realized pushing the links back into the same database was slow, so now I have a packed binary on-disk representation: 45megabytes, of the form { src: int, count: short, [ { dst: int, weight: short } ] }.
I think my first project will be shortest paths, but that will probably have to wait for another weekend. (I notice that the A* algorithm page links to a thorough and generally great page by Amit, a coworker. (And, now that I poke around, I see his PhD work was on an OO extension for ML!))
graydon's comment on my last post.)It turns out at the C level there are two interfaces for fetching MySQL results: one pulls the entire response into the client, and then satisfies repeated requests for rows pull from there, while the other pulls a row at a time. The first requires more memory, but the second locks up the database if the application is slow. The O'Caml (and I imagine other) bindings use the first. No wonder my SELECT cur_title FROM cur was slow.
I also realized pushing the links back into the same database was slow, so now I have a packed binary on-disk representation: 45megabytes, of the form { src: int, count: short, [ { dst: int, weight: short } ] }.
I think my first project will be shortest paths, but that will probably have to wait for another weekend. (I notice that the A* algorithm page links to a thorough and generally great page by Amit, a coworker. (And, now that I poke around, I see his PhD work was on an OO extension for ML!))
I won't pretend to really understand O'Caml, but I'm fairly certian that o'caml is more perlish then it is c++like.
table locking
Does the one-row-a-time thing lock all rows with InnoDB tables? Your other message mentioned using that.