From your results, I assume that your data is imbalanced. I suggest you measure your result with F1-Score instead of accuracy. From F1-score, you can get some insights about True Positive Rates (TPR) and False Positive Rates (FPR). These are important information for imbalanced dataset.
If you insist using accuracy, you should balance your data using either under-sampling or over-sampling techniques. But, I suggest an over-sampling method called Synthetic Minority Oversampling Techniques (SMOTE) to 'balance' your data.