I've used two approaches to coring mangrove sediments: a D-section ("Russian") corer, or just aluminium irrigation piping.
The D-section corer has a semi-circular barrel of around 4.5 cm diameter, and 50 cm length. Once in the sediment it rotated around a stationary blade, enclosing the sediment. Cores are retreived in 50 cm sections with deeper sediments reached by adding extensions to the handle. One advantage of this type of corer is that there is no compaction, since the sediment is effectively sampled laterally as the barrel is rotated. Another advantage is that the sediment can be easily viewed, logged and sampled in the field.
Here is an example of one: http://www.aquaticresearch.com/russian_peat_borer.htm.
I have also used aluminium irrigation tubing. The principle is simple - push the tube into the sediment, fill the top with water, seal the top, and pull out. The water and seal basically prevent the sediment from sliding out as you extract the core. This type of thing is good for sealing, http://www.cob-industries.com/nylon.aspx, or just a rubber stopper of the appropriate size. This method does cause compaction of the sediment, so it is necessary to measure (1) the tube length before insertion into the mud, (2) the length remaining above the mud once in, and (3) the depth to the sediment surface inside the tube before extraction. These three measurements enable you to calculate the amount of compaction and therefore scale up (assuming linear compaction) the cored sediments to their original depth. i think the tubes I used were 7 cm diameter and about 1.2 mm thick. You may find that you need to use a (sledge/lump) hammer to get the tube in, especially through mangrove roots, although I have mostly only needed that for sands. I collected cores from the Bahamas in this way, and got 3 m deep without needing to hammer at all. You'll need some sort of cap to take the force of the hammer if you do need one,and a hacksaw is handy for cutting off excess tubing and tidying up hammered core-tops to get the seal in. You can also sharpen the bottom of the tube with a file to help with penetration. Also, in some cases it is difficult to get these cores out, in which case some sort of clamp around the core barrel is useful. One drawback of this approach is that the cores need to be sawn open using a circular saw. You can use PVC tubing similarly if you are doing metal analyses, but these might crack under hammering or with resistant sediments/mangrove roots.
Both approaches will struggle once the sand content of the sediment is too high.