Technical note from the sysadmin

Hey, folks.  Just a note here from your system administrator.  I’m aware the site has been slow a lot lately.  I’ve made some changes to the web server configuration to better cope with some of the traffic spikes we’re seeing, so I think it should be quite a bit snappier now.  I’ll continue to keep an eye on the situation.

  18 comments for “Technical note from the sysadmin

    • Thanks. I’m actually still not happy with it — traffic to the site is really bursty and it’s still slow during some of the bursts. I’m working on upgrading the virtual host it sits on to cope with these better.

  1. Site is still sometimes slow. I’ve put up a mirror at “http://russell2.math.bme.hu/~ambrus/sc/comic/millie/” . If you (the administrator or Dana) don’t like it and want me to take it down, please email me.

    • Well, that explains why I was seeing someone trying to download the whole site last night. That was a real head-scratcher for me for a bit. I couldn’t figure out why that many non-cached pages were being hit all at once. In the future if you’re going to use automated scripts to scrape content it’d be courteous to limit yourself to one active download at a time.

      I’m curious what strips you found that were missing. I’m in the process of going through the originals and matching them up with strips in the archive, and I’ve found some missing batches of strips in that process, but I know I haven’t found them all yet.

      I’m not sure how I feel about the mirror. I guess that’s up to Dana.

      BTW, we just got a server upgrade so the site should be somewhat better now.

        • Could you tell me which of the following you don’t like in particular?

          (A) That I am hosting a copy of the comic images on my server,
          (B) that I am showing the comic images in a webpage context other than your official page,
          (C) that I show a custom table of contents, or
          (D) that I publish the fifty-something images that are not currently available from this server, such as most of 2001 October.

        • Ah, I can see the problem: your webpage now has advertisments, so

          (E) my mirror could reduce your advertizing revenue.

          I’m sorry, I usually mentally filter out the ads, so I haven’t noticed them on your webpage yesterday.

          • Honestly I don’t care that much about that. We’re not really making anything off the ads anyway; at best they help cover the hosting fees.

            Your level of snarkiness here seems a little uncalled for, frankly.

      • orv: I definitely was not downloading the whole site downloaded last night. I have downloaded all the comics in November 2009. Last night I only downloaded the table of contents a few times and some comics to check that my archive is aligned with your website, but all that was much less traffic than what you’d get if I downloaded the whole website. However, whoever was downloading all the strips might have urged me to set up the mirror site, for the slowness of this site was part of my motivation.

        In fact, if I had downloaded the comics last night, I wouldn’t have all these missing comics you’ve just asked for. You can find them on my mirror by seeing that the table of contents doesn’t have a link to this site next to them. For convenience, here are the dates for them.

        1998-05-09, 1998-09-18, 1998-10-19, 2000-07-17, 2001-10-09, 2001-10-10, 2001-10-11, 2001-10-12, 2001-10-13, 2001-10-15, 2001-10-16, 2001-10-17, 2001-10-18, 2001-10-19, 2001-10-20, 2001-10-22, 2001-10-23, 2001-10-24, 2001-10-25, 2001-10-26, 2001-10-27, 2001-10-29, 2001-10-31, 2006-04-25, 2006-06-26, 2006-11-01, 2006-11-29, 2006-12-25, 2008-01-29, 2008-02-11, 2008-02-19, 2008-03-20, 2008-04-01, 2008-04-02, 2008-04-04, 2008-04-24, 2008-05-07, 2008-05-14, 2008-06-16, 2008-07-07, 2008-08-13, 2008-09-07, 2008-09-18, 2008-09-22, 2008-12-13, 2009-01-01, 2009-01-15, 2009-03-26, 2009-05-03, 2009-07-22, 2009-08-21, 2009-09-09, 2009-09-11, 2009-09-14, 2009-09-16, 2009-10-19.

        Note that I also suspect there are a few comics that both my mirror and this site are missing, especially a few of the last ones from 2009.

        • More technical info about the missing strips. There are two different reasons here why I suspect I could be missing a strip.

          One is that it was uploaded after I scraped the website. This I think includes only one or two strips from 2009.

          The second is that the navigation links on the old site worked a bit strange, a few strips were not reachable with the “next” links under comics, so I missed those when crawling. I noticed that at that time, but I thought it would affect only a few comics from 2009 that I then fixed. I was wrong, it turns out that I’ve missed at least five more comics (namely 2006-01-12, 2007-03-05, 2007-03-06, 2007-03-07, 2007-03-08) that I’ve now found in the new TOC and downloaded last night. Thus, there could be a few more that I’ve both missed in 2009 and aren’t in your TOC now, and I have no way to recover those.

        • Also, I’ve noticed you have a different version of 2008-12-23. Thus, in case want to make sure the strips I have are the same as the ones on your server, here’s the md5 checksum of all the image files.
          “http://russell2.math.bme.hu/~ambrus/sc/comic/millie/img-all.md5”

        • Sorry, then. Someone was, and you can see why I’d assume it was you given the timing of your post. It irked me because the site was very slow that night BECAUSE it was being mirrored, so…yeah. It looked like you were complaining about a problem you were part of, to me.

        • Thank you very much for that comprehensive list.

          I’ve gone through those and fixed most of the ones that are actual strips. We made a conscious decision to not include book filler pages, vacation art, and things like that in the comic archive; I can see why a real Ozy and Millie completist would want them, though, and you’re of course free to hang on to them in your mirror. We may create another section for those at some future date.

          Also, thank you for retaining links back to the original site. I do appreciate that.

    • I have restored the mirror at “http://russell2.math.bme.hu/~ambrus/sc/comic/millie/”. That mirror has been inaccessible for about two months because the server hosting it has died.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.