File Types Used in Child URLs

We also studied the distribution of file types described in the set of extracted child URLs. We inferred the file type from the file name extensions (e.g., ``.gif'') found in the URL path. In the table below, the ``% of Docs'' column indicates the percentage of documents which contained a file of a given type. The ``# of Occurrences'' column shows the total number of extensions of the given file type that were observed. The ``# of Docs'' column indicates the number of documents which contained one or more extensions of the indicated type. Note that files can be counted multiple times, e.g., file.ps.Z would be counted as a file having both ``.ps'' and ``.Z'' extensions.

File Type and File Name Extensions
Category Type (Extension) % of Docs # of Occurrences # of Docs
Compression/Archive GNU zip (gz/gzip/taz/tgz) 0.7% 126839 18694
Zip (zip) 0.7% 157918 17277
compress (Z) 0.6% 121519 16857
BinHex (hqx) 0.3% 138259 7188
StuffIt (sea) 0.1% 5290 2615
LHArc (lha/lharc) 0.0% 20985 597
ARC archive (arc) 0.0% 432 129
Document HTML (htm/html) 76.3% 21982792 1995731
text (txt) 2.2% 325165 57476
PostScript (eps/ps) 1.8% 239949 46977
MS Word (doc) 0.2% 20153 5959
Adobe Acrobat (pdf) 0.2% 30640 5360
TeX DVI (dvi) 0.2% 14680 4163
Tex (tex) 0.1% 11998 2993
TROFF (man/me/ms) 0.1% 6488 2191
Rich Text (rtf) 0.0% 3921 1184
Maker Interchange (mif) 0.0% 262 113
Audio Sun audio (au) 0.7% 60405 18865
MS WAVE (wav) 0.3% 24361 7325
Audio IFF (aif/aifc/aiff) 0.1% 7761 2611
MIME audio (snd) 0.0% 1839 600
Amiga MOD (mod/nst) 0.0% 4202 254
IRCOM (sf) 0.0% 353 161
IFF (iff) 0.0% 322 47
SoundBlaster (voc) 0.0% 122 27
U-law (ul) 0.0% 21 19
FSSD (fssd/hcom) 0.0% 3 3
Image GIF (gif) 61.7% 9990239 1614244
JPEG (jpe/jpeg/jpg) 7.8% 811353 205088
X bitmap (xbm) 2.9% 968410 75825
TIFF (tif/tiff) 0.2% 22546 5416
X pixmap (xpm) 0.0% 3448 814
RGB (rgb) 0.0% 985 259
portable pixmap (ppm) 0.0% 646 124
portable graymap (pgm) 0.0% 219 78
portable bitmap (pbm) 0.0% 114 70
X window dump (xwd) 0.0% 277 66
raster (ras) 0.0% 221 54
portable anymap (pnm) 0.0% 51 7
Movie MPEG (mpe/mpeg/mpg) 0.3% 21496 7460
QuickTime (mov/qt) 0.2% 15026 5199
MS video (avi) 0.1% 5589 1742
SGI (movie) 0.0% 538 313

Category	Type (Extension)	% of Docs	# of Occurrences	# of Docs
Compression/Archive	GNU zip (gz/gzip/taz/tgz)	0.7%	126839	18694
	Zip (zip)	0.7%	157918	17277
	compress (Z)	0.6%	121519	16857
	BinHex (hqx)	0.3%	138259	7188
	StuffIt (sea)	0.1%	5290	2615
	LHArc (lha/lharc)	0.0%	20985	597
	ARC archive (arc)	0.0%	432	129
Document	HTML (htm/html)	76.3%	21982792	1995731
	text (txt)	2.2%	325165	57476
	PostScript (eps/ps)	1.8%	239949	46977
	MS Word (doc)	0.2%	20153	5959
	Adobe Acrobat (pdf)	0.2%	30640	5360
	TeX DVI (dvi)	0.2%	14680	4163
	Tex (tex)	0.1%	11998	2993
	TROFF (man/me/ms)	0.1%	6488	2191
	Rich Text (rtf)	0.0%	3921	1184
	Maker Interchange (mif)	0.0%	262	113
Audio	Sun audio (au)	0.7%	60405	18865
	MS WAVE (wav)	0.3%	24361	7325
	Audio IFF (aif/aifc/aiff)	0.1%	7761	2611
	MIME audio (snd)	0.0%	1839	600
	Amiga MOD (mod/nst)	0.0%	4202	254
	IRCOM (sf)	0.0%	353	161
	IFF (iff)	0.0%	322	47
	SoundBlaster (voc)	0.0%	122	27
	U-law (ul)	0.0%	21	19
	FSSD (fssd/hcom)	0.0%	3	3
Image	GIF (gif)	61.7%	9990239	1614244
	JPEG (jpe/jpeg/jpg)	7.8%	811353	205088
	X bitmap (xbm)	2.9%	968410	75825
	TIFF (tif/tiff)	0.2%	22546	5416
	X pixmap (xpm)	0.0%	3448	814
	RGB (rgb)	0.0%	985	259
	portable pixmap (ppm)	0.0%	646	124
	portable graymap (pgm)	0.0%	219	78
	portable bitmap (pbm)	0.0%	114	70
	X window dump (xwd)	0.0%	277	66
	raster (ras)	0.0%	221	54
	portable anymap (pnm)	0.0%	51	7
Movie	MPEG (mpe/mpeg/mpg)	0.3%	21496	7460
	QuickTime (mov/qt)	0.2%	15026	5199
	MS video (avi)	0.1%	5589	1742
	SGI (movie)	0.0%	538	313