If the remove button isn't working, then it's the right manner yes. AdwCleaner is still portable even if it creates some files inside C:\AdwCleaner.
Keep in mind tho that the folder can contain Quarantined files that you won't be able to retrieve after removal. So I would still recommend keeping the C:\AdwCleaner folder.
Regards,