JPEG (Joint Photographic Experts Group) is a very efficient, true-color, compressed image format. It uses lossy compression, which means that bits are removed from image in order to save space. JPEG files support millions of colors (compared with 256 for GIF).
The JPG format is best for images with gradients, blends, and inconsistent color variations such as photographs or paintings. Images which have well separated tones should be saved in GIF or PNG format. For example, if you include text in your image, you will notice a definite fuzzing of characters when you view it.
You have several options when optimizing JPG images. JPG is a lossy format (which means it throws away bits), so with each generation of image you save you will loose information, and that will degrade quality of image.
Any good image editor (Photoshop, Paint Shop Pro and others) will allow you to specify a compression percentage when you save file. Before you purchase a graphics editor make sure it has a preview pane on save so you can see what compressed image looks like when you save it. This is a lot easier than saving a copy, examining it, resaving it and so on.
JPEG supports a concept similar to interlacing in GIF which is called progressive JPEG. This simply means that a rough image is displayed initially, followed by more and more detail as additional bits of image are received. This is good for displaying large images. Unfortunately, progressive JPEG is a relatively new standard and is not supported by all browsers.
There is a new format which is threatening supremacy of GIF and JPEG. That format is called PNG (Portable Network Graphics).