Yes, residual stress affects the overall factor of safety. if the residual stress is tensile and your component is subjected to tensile loading then it will decrease the life of your component whereas if the residual stress is compressive and your component is subjected to tensile load then the load has to overcome the compressive effect hence the life of the component increases.So it depends upon nature of the residual stress which will decide the positive or negative impact. Different destructive,semi-destructive and non-destructive techniques can be used to find residual stress (for details refer to research papers). You can also follow the recommendations provided by IIW (International Institute of Welding).   

Residual stresses have usually roughly the intensity of the thermal gradient multiplied by the coefficient of thermal expansion. For welding needless to say thay are important with fusion processes. If defect is present within the structure, it will surely be influenced by existing tensile residual stresses. For more details, you can look at some of the publications here (pre-2006) :

https://www.researchgate.net/profile/Daniel_George8/publications?sorting=newest&page=2

Regards

Residual stresses will affect the overall safety factor for sure. You could make reference to ASTM specifications to understand the requirements

https://www.astm.org/BOOKSTORE/COMPS/WELDING15.htm

or you can have a look here for some calculations

http://www.aisc.org/uploadedfiles/steel_solution_center/technical_resources/engineering_faqs/2005specification_third_printing.pdf

If you have a background in finite element methods, you could carry out an investigation running welding numerical simulations of your structure to realise the residual stresses involved and from there take decisions based on your application. e.g. you apply to your structure a cyclic load, a static load? What level of risk is acceptable for your application?

As Daniel George said, if in your welded joint you have defects these are going to affect the reliability of your structure. So, also the quality of your welding process should be taken in consideration.

Yes residual stress is a factor in safety

as a rule of thumb  the last metal to solidify  is in tension,  if that area is at a stress raiser like a corner or  a fillet that will be under tension  in service, the combination can lead to a early failure. Consider a stress relief treatment or a process change to minimize the situation

Safety factor (SF) is a global parameter used to ensure that the design fill the requirement during service life. The safety factor lowers the working field (0 - Rp0,2) to (0- Rp0,2/SF).

Welding residual stresses are induced by contraction strain (you have tensile stresses). The tensile stress adjacent to the weld line are as high as Rp0,2 of the base metal. That means, you have no safety for the welded parts when the service (tensile) stresses is applied at the same area (volume) and then you have to release the residual stresses before service! I mean, no safety factor can help.

BUT when the service stresses are compression stresses then you can use the usual safety factor recommended for the type of design and service conditions. I mean, the base metal is then relevant and not the weld itself.

When the weld is partly affected by the load during service, then you have to know the type of load and amplitude to be able to evaluate the situation.

The best method to evaluate this is the FE-method for both welding and service. Then you can ensure whether the residual stress must be considered or not. In addition, you have to do some FE (service-life) calculation with relevant tests if the loading is dynamic.

Yes you must consider residual stresses, however, to determine the FoS, the magnitude is essential. In general, the welding power and other process parameters such as travel speed will determine the extent of residual stresses.