Diary of a Webster

Mac Maintenance

Here's Apple's recommended maintenance:
Mac Maintenance Quick Assist

Keep Things Up to Date
Make it a point to check for Apple software updates once a month
No problem, I have it set to check automatically
Put Your Files Away
Put your files away where you can easily find them.
A question about email attachments: should I just keep the email and attachment, or save the attachment and delete the email?
My habit has been to do both: if I sure I want the attachment, I will save it to a sensible location.
The problem comes with attachments that I can't use immediately, but don't want to throw away either. I have at least 150 M of such stuff.

404s fixed

We moved a folder:

/training/reacademy

became

/reacademy/

but search engines are still looking for the old URL.

How to fix this?

1. create a text file called /training/reacademy.php that contains:


<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"
       "http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Language" content="en-us">
<meta http-equiv="content-type" content="text/html;charset=iso-8859-1">
<title>Tech Transfer Resources</title>
<meta http-equiv="refresh" content ="0; URL=http://www.techtransfer.berkeley.edu/reacademy/">
</head>
<body bgcolor="#ddeeff">

<h1>Please Update Your Bookmark</h1>

<p>
The page you accessed has been moved.
If you are not automatically redirected to the new location,
please click <a href="/reacademy/">here</a>.
Please update your bookmarks or favorites.
</p>
</body>
</html>

But, this gives log lines like:


"GET /training/reacademy HTTP/1.1" 200 673 "-"

If we want the crawlers to know the URL is no good, we need to send a 301 - Moved Permanently, so this is better:


<?php

header("Location: http://www.techtransfer.berkeley.edu/reacademy/",TRUE,301);

exit();
?>

Which gives:


"GET /training/reacademy HTTP/1.1" 301 5 "-"
"GET /reacademy/ HTTP/1.1" 200 23730 "-"

(According to w3.org:

Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).

So I also included the html as above, minus the "refresh" META tag.)

common error log entries

Of the latest 4444 lines in error log:

533 mod_ssl: SSL handshake failed
449 sslv3 alert certificate expired
I don't think we run any pages over https
3212 File does not exist:
- 1,623 /freestuff/* almost all from bots
- 180 co.tulare.ca.us requests for clearinghouse/images/*
- 180 ci.stockton.ca.us requests for workzones/images/*
- 180 vwar (vulnerability probes)
- 171 /sitemap/* all fixed, crawler indexes haven't updated?
- 135 /https:/registration.techtransfer.berkeley.edu/ all from client 87.118.108.229
- 134 cp2info/ 74 from googlebot, 57 for favicon.ico.
- 82 search2.Berkeley.EDU
- 73 phpMyAdmin and ilk (vulnerability probes)
- 58 pavementpres06downloads1 (from googlebot)
- 50 bad clearinghouse URLs (fixed today)
- 4 case-sensitive filenames in newsletter/00-2 (fixed today)

of the latest 99,999 access log lines, 391 (.04%) are 404:

Noteworthy:

1 reqs for aviation05downloads/Joselzon.pdf, which we pulled for IP reasons
1 bad link from http://rspcb.safety.fhwa.dot.gov/FullEventInfo.asp?EventID=64&CallingPageID=search&SearchRequested=True&PageFormat=html&TopicID=11
(I filled out a form at FWHA's RSCBP site to fix this one)

122 squirrelcart probes
73 phpMyAdmin etc. probes
63 from search2.Berkeley.EDU: 42 bad /sitemap/ URLs
27 from 87.118.108.229, a german crawler?
22 vwar probes

Provisioning new users

This is an ongoing list of things to do when we hire someone:

set up an account on the file server
- to reset password, as root, type
```
[root ]# passwd <username>
```
- to reset password, as root, type
```
[root ]# smbpasswd <username>
```

Labels: file server, file share, new users, password, provisioning, samba, users

Price added to freestuff item?

I lurk on most outward-facing email lists for our department. Today I saw a confirmation email from G3 (free stuff) with this:
SKU18 Inspector's Job Guide and Highway Maintenance Tables $6.00 6 $36.00

I emailed the list to point this out.

Labels: freestuff, g3

ssh tunnel

I always have to look this up:

ssh  steve@www.techtransfer.berkeley.edu -L 8888/localhost/8800

The parts:

ssh: the secure shell command
user@host: your account on the host to which you want to tunnel
-L: ssh forwarding (tunneling) flag
8888: local port (could be 8800 in this case, just needs to be unused on your machine)
localhost: your machine
8800: port on remote host

then in (for example) your browser, you can enter the address

127.0.0.1:8888

and get the page at

www.techtransfer.berkeley.edu:8800

Labels: clearinghouse admin, mediaLibrary admin, port forwarding, server admin, ssh, ssh tunnel

fixed 404s

[Mon Nov 3 11:18:58 2008] [error] [client 169.229.192.166] File does not exist: /usr/local/apache/htdocs/newsletter/01-4/2001-4.pdf
FIXED: -> 01-4.pdf

[Mon Nov 3 11:21:37 2008] [error] [client 169.229.192.166] File does not exist: /usr/local/apache/htdocs/aviation/2003/download.php
FIXED: -> /aviation2003downloads/
in /aviation/2003/index.php

also:
/aviation2000downloads/
FIXED: -> /aviation/2000/
in /aviation/2004/index.php, /aviation/2005/index.php

[Sun Nov 2 01:57:47 2008] [error] [client 169.229.192.166] File does not exist: /usr/local/apache/htdocs/sitemap/2004techtopics.pdf
[Sun Nov 2 01:57:47 2008] [error] [client 169.229.192.166] File does not exist: /usr/local/apache/htdocs/sitemap/2002techtopics.pdf
[Sun Nov 2 01:57:50 2008] [error] [client 169.229.192.166] File does not exist: /usr/local/apache/htdocs/sitemap/2006techtopics.pdf
[Sun Nov 2 01:58:17 2008] [error] [client 169.229.192.166] File does not exist: /usr/local/apache/htdocs/sitemap/1999techtopics.pdf
[Sun Nov 2 01:59:22 2008] [error] [client 169.229.192.166] File does not exist: /usr/local/apache/htdocs/sitemap/1998techtopics.pdf
[Sun Nov 2 01:59:22 2008] [error] [client 169.229.192.166] File does not exist: /usr/local/apache/htdocs/sitemap/2005techtopics.pdf
[Sun Nov 2 01:59:23 2008] [error] [client 169.229.192.166] File does not exist: /usr/local/apache/htdocs/sitemap/2001techtopics.pdf

fixed in: /sitemap/index.php

Newsletter

I also fixed a LOT of broken links in older newsletter folders (.html instead of .php, old paths that were replaced 3 or 4 years ago...)

Newsletters from late 2006 and all of 2007 didn't have index.php pages, so I copied HTML from the newsletter/archive page into new index pages for each issue in 2007. Still need to do 2006.

Labels: 404, Apache log, error logs, maintenance, web server logs, web site

robots.txt

Added a line:

Disallow: /freestuff/ # use /g3/ instead

Labels: robots, robots.txt, server admin, System Administration, web site

Diary of a Webster

Wednesday, December 03, 2008

Mac Maintenance

Wednesday, November 19, 2008

404s fixed

Monday, November 10, 2008

common error log entries

Of the latest 4444 lines in error log:

of the latest 99,999 access log lines, 391 (.04%) are 404:

Wednesday, November 05, 2008

Provisioning new users

Tuesday, November 04, 2008

Price added to freestuff item?

ssh tunnel

Monday, November 03, 2008

fixed 404s

Newsletter

Friday, October 31, 2008

robots.txt

About Me

Links

Previous Posts

Archives