The simplest way to get rid of the water in the oleylamine is to dry over 3A molecular sieves. Subsequent to drying over molecular sieves take an NMR spectrum to see how dry your sample is and additionally a mass spec too would be useful.
As oleylamine is of high bp it may be pretty difficult to distil unless you use vacuum distillation. But I would try drying first and then take an NMR to find out how pure the sample is before distillation.
You could also simply dry the oleylamine under high vacuum (1 mbar or less) with no heating or heat it only mildly (to 30-40 degrees Celsius), below its boiling point at the pressure which you would use. You culd estimate the boiling point under reduced pressure using online programs like this one: http://www.trimen.pl/witek/calculators/wrzenie.html
For thorough drying, you would need two hours or more, it is good to record NMR spectrum after drying, as Vidhura Mahendra mentioned.