Alponsine - the simple fact is there is no difference. They are the same thing. Adding 'system' would not be that common - but you could do it for many disciplines i.e. health care and health care system; they are the same. Some semantics might argue 'one occurs within the other' i.e. health care occurs within a health care system - or public health occurs within a public health system - but they can't be separated at the end of the day - so the system is merely an 'ethereal' context that isn't really needed.
The aim of Public Health is to identify and evaluate the epidemiology, research over the disease and make out the optimum prevention rather than treatment. Actually it controls the health status of the entire population of a society.
On the other hand, Public Health System is the logistic management system that helps to improve the gross health status of the population.
That is why we must say that, Public Health System is a part of Public Health.
You can have a check the following url for better understanding.
My simplistic way to think about this it that Public Health is a population-based approach to improved community health: it is primarily a strategy informed by epidemiological data. The Public Health System, a network of public and private organizations, is responsible for implementation of the strategy: it is primarily focused on tactics supported by process and outcomes data.