First of all, make sure that you have checked the box that says: "These people gave permission to be added to my list". Also check that your data file is in the right format, as shown on the subscribers list upload page.
Important! The subscriber list data file must not contain any empty rows. If it does, all the of records below the empty row will not be imported.