Download a suitable icon to use and ftp to the images directory of your template

The one I used was here:

Add the following code to your template.css file:

a[href^="http://"] {
background: url('../images/external.png') center right no-repeat;
padding-right: 13px;
background: none;
padding-right: 0;

The second part will eliminate the icon for internal links.