Adam Jackson (ajaxxx) wrote,
  • Music: nine inch nails - gave up

wtf pthreads

So I had this code that did essentially:
for (int i = 0; i < num_files; i++)
    if ((fd = open(files[i]->name, O_RDONLY)))
        do_stuff(fd);
And I figured, hey, I've got an I/O scheduler and my disks have command queueing now, let's mitigate some of that seek death and do multiple stuffs in parallel. Well, here in ***america*** where fork() is cheap you'd do:
for (int i = 0; i < num_files; i++) {
    if (num_kids == MAX_KIDS)
        wait(NULL);
    if ((fd = open(files[i]->name, O_RDONLY))) {
        if (!fork()) {
            do_stuff(fd);
        } else {
            num_kids++;
        }
    }
    while (waitpid(-1, NULL, WNOHANG) > 0)
        num_kids--; 
}
while (num_kids--)
    wait(NULL);
And then it's off to the bar. For some reason I thought that maybe doing it in pthreads would be a good exercise, and tried it that way first. (Actually I tried POSIX AIO zeroth. Ha! Good one, guys. You almost got me.) That went something like:
for (int i = 0; i < num_files; i++) {
    if ((fd == open(files[i]->name, O_RDONLY)))
        pthread_create(&thread[min_thread_index], NULL, do_stuff, (void *)fd);
    /* hmm, how to throttle thread creation? */
    pthread_join(&thread[i % MAX_KIDS], NULL); /* bonghits in the hood */
    /* and then reset min_thread_index somehow? ugh */
}
Except that doesn't work, because you can only join with named threads, and not just whichever one happens to die first. I have no idea how long each of these threads is actually going to take, so round-robin is a waste of time. I could stick in a semaphore that every thread downs on init and ups on termination, and block the dispatch loop on that, but that's still not enough, since then I still need to figure out - somehow - which thread died so I can reuse its slot in thread[]. Why pthread_create() doesn't just return the new pthread_t is beyond me.

Of course ideally we'd have waitfd() that was smart enough to cope with thread IDs too, and you'd message the thread ID back out to the master process somehow, and you'd still have to allocate your own pthread_t's anyway. Hooray for unix.
Tags: fail, nerdery
  • Error

    default userpic

    Your reply will be screened

  • 12 comments