There are several books on Kalman filtering and integrated navigation that would give you the answers, at least in part--This book by Paul Groves is the state of the art:

http://www.amazon.com/Principles-Multisensor-Integrated-Navigation-Applications/dp/1608070050/ref=sr_1_2?ie=UTF8&qid=1418668032&sr=8-2&keywords=integrated+navigation

and the book by Britting is the classic on inertial integration:

http://www.amazon.com/Inertial-Navigation-Analysis-Technology-Applications/dp/1608070786/ref=pd_bxgy_b_text_y

The Kalman filter is a state observer (or Luenberger observer) or estimator that minimizes the variance of the state estimate obtained from measurements. It is able to do so in real time because it is implemented recursively--it doesn't take very much of computing or memory in each step--relative to what a batch optimization method would take for example.

Thus, the Kalman filter simply takes the various measurements in a suitable coordinate frame as output equations (e.g., y(k)=Cx(k)+D1u(k) +v) with measurement noise v, and estimates the state of the system x(k+1)=Ax(k)+B1u(k)+B2w(k) with state x and process noise w. This is usually done in the body frame of the aircraft, and then transformed suitably to LLH (latitude longitude, height), NEH (north, east, height with some local origin), or ECEF (earth centered earth fixed--usually used in GPS and satellites)  coordinates to aid the flight control and flight management systems. The state of the aircraft, or most any vehicle, is typically its 3 components each of position, velocity, attitude, and angular velocity.

The execution time for a Kalman filter on a commercial aircraft was under 5 milliseconds even in the 1990s. It assumes that the governing dynamics of the body are linear differential or difference equations, and that the measurement noise and process noise are normally distributed.

These are strong assumptions, and certainly not true for an aircraft, whose dynamics are nonlinear--as with any rigid body governed by the Euler equations. However, most aircraft operate in the regime of linear dynamics and linear aerodynamics, i.e., they mostly fly at small angles of attack in ascent or descent, and mostly straight and level. While measurement noise is normally distributed in many cases, especially when produced by circuitry (Johnson noise, shot noise and flicker), process noise is seldom well characterized, and depends upon the context, and the aspects of dynamics we choose to ignore. However, when when high sampling frequencies are available for measurements, vehicle dynamics can be approximated by simpler models to great effect.

Agree with prior answer recommending Groves and the comments about the Kalman filter.  The hard part is knowing what the measurement errors are, including biases and noise.  Measurements from some sensors are better matched to a given situation than others.  The measurements innovations tell you which measurements are consistent with a trend, which is useful for identifying outliers, but they don't tell you which measurement trends are trustworthy. In a geomagnetic storm, magnetic compass measurements may not show much of a trend at all.  Interferrenc from a nearby "hard iron" source may show a consistent but totally erroneous tend.  Sorting out these considerations requires a higher level of situation awareness than a Kalman state estimator. It requires some form of Data Fusion algorithm.  There are several text books on data fusion methods and many papers dealing with applications to multi-sensor integrated navigation.

try to understand KF as a parameter estimator. modelling involves matrices with coefficients and noise model(normally assumed to be white noise). this is only the starting point.

See:

http://jameslfarrell.com/tracking/kalman-filter-or-suboptimal-does-it-matter

The correlations are performed based on the information each system gives.For example if you want to improve your position you can combine INS and DME. The INS gives the present position in lattitude and longitude. The DME provides you with the range from a station. With just this information is not possible to calculate your position, but knowing the station position and previous position estimation you can estimate the DME range error.  Next time the DME provides the range information you can predict  your  correct position. The difference of this "corrected" position and the INS output is the INS error. So KF is a recusive algorithm. You can use the error dynamics of the INS, which is very low for a small period, in order to use its correct data until the next DME data to update your error estimation. We can update the INS data at 1s, that give you a lot of calculation time. In the case you use VOR/ILS you use the bearing measured and bearing estimated to predict the errors. For INS and GPS an update frame of 1 minute is used.