It all depends on the method of obtaining the mass spectrum, for example this database will be useful. 

http://www.massbank.jp/SearchPage.html

Identifing an completely unknown only by MS is difficult, even if you have high res and MS/MS data.If you have a high resolution mass spectromter it will usually provide you the formula within the software (just check the manual). the lower your mass acccuarcy the more formulas are possible.

Anyway for searching, I suggest the folowing databases to start with.

http://webbook.nist.gov/chemistry/

http://metlin.scripps.edu/index.php

Thanks Christian Berchtold,

I used a HR-LC-MS and got a mass of my compounds like 338.25168 Da and 524.28254 Da. From this data, NIST library search revealed 68 match compounds and 8 match compounds. How do I sortlist the most relevent compound and how can I identify my compound is novel or not?

There is many solutions for your problem.

for a repos review you can also look to : https://www.researchgate.net/publication/267467312_Current_approaches_and_challenges_for_the_metabolite_profiling_of_complex_natural_extracts

anyway if if you have match in NIST database you can primarily think that your compound is already known...

any way you then need to do some bibliographic search to confirm it!!!

if you just want to confirm the molecular formula look to the "seven golden rules" from fiehn lab! (Easy to find in Google)

Article Current approaches and challenges for the metabolite profili...

You received many valuable answers for sure.

In your case if you assume that you get a new compound, just draw and calculate the suggested formula as CHN.... etc then follow the available link

http://www.sisweb.com/referenc/tools/exactmass.htm

you can then compare the exact mass of the suggested formula with that experimental.

But then you need to use tools of NMR and other tools to proof your suggested structure

If you found the compound in a library, then the discovery of this compound is  not novel (since somebody has already identified it and added it to the library).

It could be that you have located an isomer of said compound but one can not determine this by MS1 alone. It is possible that MSn can help you eliminate additional candidates until a single  candidate emerges. 

Also, there are various tools for determining possible empirical formulae for a given mass, maximum mass error, charge, elemental constraints, and ionization methods.

A couple of such programs can be found at:

https://nationalmaglab.org/user-facilities/icr/icr-software Select "Molecular Formula Calculator"

http://www.chemcalc.org/mf_finder 

Many other labs have similar tools.

Unfortunately, most of the required information is missing from this discussion to this point.

http://www.ncbi.nlm.nih.gov/pubmed/22497521