This is a very complicated ligation reaction which is not very favorable to obtaining your end product. In particular, I think the limiting steps would be the ligation of the PacI ends (only a two nucleotide AT overhang, a very weak sticky end) and the NaeI ends (blunt end, blunt end ligations are much less efficient than sticky ends). The large size of the end product is also a limiting factor.

As I see it, you have a few options:

1. Switch your cloning strategy to one which is better designed for assembling fragments from multiple inserts such as Golden Gate or Gibson Assembly. If you use Golden Gate, you have to double check and make sure the Type IIS enzyme(s) don't internally cut your fragments or your vector, if they do it may still work but you would have to modify the procedure and make sure that all sticky ends are unique.

2. Use multiple cloning steps with transformations instead of doing it all in one go. You could ligate fragments 1/2 and 3/4 separately into their own unique plasmids (preferably a high copy one, if possible), transform these separately, verify those clones, then extract the 1/2 plasmid and the 3/4 plasmid and cut your fragments out of them, then attempt to do your ligation into your intended plasmid backbone. That would simplify the final ligation from a reaction between 5 separate DNA fragments (4 inserts and vector) to only 3 (1/2 insert, 3/4 insert, and the vector). Importantly, this would also eliminate the inefficient PacI and NaeI ends since they would already be pre-ligated.

3. PCR amplify the entire correct 1/2 ligation product and 3/4 ligation product using primers, then digest the PCR products and do a 3-way ligation similar to #2. Be sure to use a proof reading polymerase that can handle longer fragments and add 5-6 bp of "filler" DNA at the 5' end of your primer since many restriction enzymes cut inefficiently if there's no extra DNA on one end of their cut site. I usually use AAAAAA as the filler sequence on all primer ends since it doesn't increase the primer melting temperature as much and since all the ends are identical it lessens primer dimer formation.

More info on Golden Gate cloning and Gibson Assembly:

https://blog.addgene.org/plasmids-101-golden-gate-cloning

https://blog.addgene.org/plasmids-101-gibson-assembly

As far as your transformants are concerned, I would guess they just contain the wild type vector re-ligated to itself or undigested vector. They may also contain any plasmid you used to generate the inserts, if any of those plasmids contain the same antibiotic resistance as your backbone plasmid. In order to reduce this you have to do extensive digestions and de-phosphorylations of your vector.

Preferably, you should also gel extract the linearized and de-phosphorylated vector using a kit. Gel extraction is unfortunately also very difficult for a lot of people, but my tips are:

1. Use blue light illumination with a blue light excitable DNA dye like SYBR Safe instead of UV illumination with ethidium bromide. UV damages the DNA and you have to work so quickly to avoid damage that it often ends up with lots of extra agarose to dissolve. Using blue light doesn't damage the DNA and you can really take your time cutting a nice tight square around the bands, which minimizes the amount of gel dissolving buffer needed (and thus contaminants that mess with the 260/230).

2. Most kits use a de-salting buffer wash step right before the elution step, which is usually just 70-80% ethanol with the remaining 20-30% being an aqueous buffer. Do this de-salting step 3 times and leave the buffer on the column for 5 minutes each time, then make sure the column is thoroughly dry before adding the elution buffer (Qiagen kits in particular are very bad about ethanol retention because of their column design).

3. Heat the elution buffer to 50-60°C before adding it to the column. Allow it to dissolve the DNA off the membrane for 5 minutes before spinning.

4. This is actually specific to your case: Most kits have you weigh the gel slice before dissolving it to know how much gel dissolving buffer to add. Once you know what "1 volume" of your gel is, after you dissolve the gel slice, add 1 volume of endonuclease free water. Adding 1 volume of water usually helps with fragments >10 kb. Example: If your gel slice ways 100 mg and the kit tells you that 100 mg of gel = 100 µL of volume= "1 volume", and you need to add "3 volumes" of gel dissolving buffer per "1 volume" of gel, you would add 300 µL of dissolving buffer. Then after that you would add 100 µL of nuclease free water. If the kit asks you add ethanol or isopropanol, do that as you normally would after the water addition.

Sorry, I know this is long, but I hope it's helpful!