Depends on how bright your bands are. In general, load ALL of it since it's better to have lots.
Key for success, minimize the size of the agarose slice. Don't use more than the column can handle, and make sure the agar is COMPLETELY dissolved before loading onto the spin column.