or in the Address bar when you return to that site via your bookmarks?
The technique only works with IE5 (Internet Explorer 5) and involves the use of a file called favicon.ico.
and then use the bookmark to come back to this page to see an example.
Here's how to set it up for your own pages.
You need to create a file called favicon.ico which is a Windows icon file. This should be an image 16 x 16 pixels that uses only sixteen colours.
If you're desperate you can use Paint which comes with Win9x, but I prefer PaintShopPro http://www.jasc.com/ There are some Freeware image editors at http://www.freewarehome.com/graphics/gredit.html. Save the file as a favicon.bmp or favicon.gif [it doesn't matter] and then use IrfanView http://www.irfanview.com/ to convert the bitmap [favicon.bmp or favicon.gif] to a Windows icon file favicon.ico. Or save it directly as favicon.ico if you have that option.
Alternatively use a dedicated icon editor to combine the two steps above. PC Magazine's IconEdit32 v1.0 ftp://zdftp.zdnet.com/pub/private/sWlIB/graphics_multimedia_tools/icon_tools/iedit32.zip is Freeware and appears adequate for the task.
You should now have your favicon.ico icon file. Mine looks like this 
You now need to upload favicon.ico to the root of your homepage directory. For Freeserve users this is / . For other ISPs this may be /pub or /ht_docs or some other. It's the place where your index.html [or equivalent] for the whole site resides.