A few things can cause mass balance error. One is truncation of minor components. It can be changed to higher precision or gradual cut off.
Another thing to check is stream connections. A flowing stream connected to a pseudo stream may run to completion but give large errors in mass balance. Pseudo streams are like tiny sample points with data transferred but not for flowing streams.
You may also check the molecular weights of the species involved (in the database), as Aspen databank components data are sometimes slightly different from the data you have from literature. Then manually check the balance using the values you retrieved from the database.
Do you have recycle streams in your process? Sometimes when you have recycle you need to ask to reconcile the streams.
Another thing to do is to increase the acceptable range of error. (Simulation – Setup- Calculation Options- Check results).