AFAIK the health effects of carbon nanotubes are still relatively poorly understood, but I certainly recommend taking precautions to avoid breathing the powder, particularly if you work with large quantities.
I personally work in a fume hood whenever possible, however this may make working with the powder difficult as it tends to get airborne due to the flow of air. To that end, a laminar flow cabinet may be more practical if you must e.g. spoon the powder for the purposes of weighing. Another possibility is to wear a type of a face mask (although most of them are not sufficient for protection against CNTs).
Finally, while breathing the powder may be the most critical route of entry, I would also wear protective gloves (even if the solvents and other chemicals you need are not hazardous).