There are commercial live/dead staining kits (such as the one from sigma at the link below). You can also stain using DAPI or Hoechst stains which stain DNA in live cells. These should not affect viability of the cells so you can then do other things with them later. Evans blue dye also penetrates dead cells so can be used. I am not certain that these will work on diatoms as they may not penetrate the cell coat. You might also need to be careful of the epifluorescence of the chlorophyll in the chloroplast, which may have overlapping absorption/emission, masking the signal.
Alive diatoms usually have brownish plastids, some of diatoms are also moving. So one could do a rough estimation using light microscope. Green plastids are typical for dying cells.
Often dead diatoms condense or leak out their chloroplasts which could be used as an indicator. The use of a dye like trypan blue that passes through cell membranes of dead cells, and color them blue, could be used. However this is a dye exclusion method since the live cells are not being stained.
There are several fluorescent dyes that you can use in determining if a cell is live or dead: see Peperzak & Brussaard Flow cytometric applicability of fluorescent vitality probes on phytoplankton J. Phycol. 2011. A probe such as FDA can easily be scored live or dead with a microscope too.
i share with you a protocol for determining viable microorganisms on cell cultures, i used for determining the viability on Dunaliella tertiolecta and Chaetoceros calcitrans (a diatom) and it works very well!!