Feature is mechanism that SharePoint used to deploy customized components. The built in components like document list, team site also deployed using feature. So basically, everything can be deployed as feature, for example list definition, content type, site column, workflow and etc, except a site template. The core of the it are two files feature.xml and elements.xml.
When you deploy features, you put all your files into the a folder, and organize the folder with this structure, CONTROLTEMPLATES, FEATURES\FEATURE_NAMES, IMAGES. The content of the folder will copied to the Template folder(c:\program files\common files\microsoft shared\web server extensions\12\Template).
The elements.xml has lots of element. But one of common use is deploying files by using Module element. You can deploy files like the following.
Microsoft.SharePoint, Version=188.8.131.52, Culture=neutral, PublicKeyToken=71e9bce111e9429c Microsoft.SharePoint.WebPartPages.ContentEditorWebPart Yet Another Web Part is Born TitleBarOnly This Web Part was added through declarative logic]]> Microsoft.SharePoint, Version=184.108.40.206, Culture=neutral, PublicKeyToken=71e9bce111e9429c Microsoft.SharePoint.WebPartPages.ImageWebPart None Watch My Gears Run /_layouts/images/GEARS_AN.GIF]]>