On the main pages, this will link to the main site feed, whereas on post pages, this will link to the comments feed for a particular post.
Here is how this can be achieved:
1. Go to dashboard/ template/ edit HTML
2. Click the 'Expand Widgets Templates' box
And delete it!
To be sure this has worked, you can preview your blog template before saving it: the “Subscribe to: Posts (Atom)” link should now be removed from your layout, on post pages as well as the main page.